Aether Project on Outlier: Pay, Requirements and How It Works
Everything you need to know about the Aether Project: RLHF evaluation tasks, hourly rates, workflow stability and quality requirements.
Among the active projects on Outlier, Aether is the one that comes up most in community discussions — on Reddit, in Slack workspaces, in Telegram groups dedicated to remote AI work. The reason is straightforward: it’s one of the few projects that offers a consistent task flow, pay in the mid-to-high range, and structured enough guidelines to make the work predictable. It’s not open to everyone, which makes it more sought after.
What the Aether Project Is
Aether is an RLHF (Reinforcement Learning from Human Feedback) evaluation project commissioned by one of the major companies developing commercial language models. The goal is to collect high-quality human evaluations to refine model behavior across real conversational scenarios.
The task categories on Aether cover a broader range than generic annotation projects:
- General conversation: evaluating responses to open-ended questions, explanation requests, discussions on complex topics.
- Logical reasoning: analyzing syllogisms, causal chains, elementary mathematical deductions. Advanced mathematics isn’t required, but you need to recognize when a logical chain breaks down.
- Light coding: verifying that code snippets are syntactically correct and produce the declared output. You don’t need to be a senior developer, but reading basic Python or JavaScript helps.
- Instruction adherence: verifying that the model followed every explicit constraint in the prompt — length, tone, format, exclusions requested by the user.
How a Typical Task Works
A standard Aether task always follows the same three-phase structure.
1. Analyzing the User Prompt
Before looking at the model’s responses, read the original user prompt with surgical attention. Identify:
- What is the core request (what does the user actually want)?
- Are there explicit constraints (length, language, format, exclusions)?
- Is there an implicit intent the model was supposed to pick up on?
This is the step most annotators skip or rush through. If you haven’t fully understood the prompt, you can’t accurately evaluate either response.
2. Side-by-Side Comparison: Response A vs Response B
The heart of the task: you’re shown two model-generated responses to the same prompt. You must choose which is better — or declare they’re equivalent, if that’s genuinely the case — and write an argued justification.
The judgment criteria Aether weights in order of priority:
| Criterion | Weight | What to evaluate |
|---|---|---|
| Factual accuracy | High | Are the claims verifiable and correct? |
| Constraint adherence | High | Did the model respect every prompt restriction? |
| Completeness | Medium | Does the response actually cover what was asked? |
| Safety | Medium | Does the response avoid potentially harmful content? |
| Style and readability | Low | Is the response clear, well-formatted and appropriately toned? |
A common mistake is inverting these priorities: heavily penalizing a response for stylistic reasons when both responses contain a factual error — that’s the problem to address first.
3. Writing the Justification
The justification is what separates a 3-star annotator from a 5-star one. Aether’s guidelines require justifications that:
- Quote or reference specific phrases or sections of the responses (not vague descriptions).
- Explain why something is problematic, not just that it is.
- Follow the criteria hierarchy — factual issues first, then adherence, then style.
Example of an insufficient justification: “Response B is better because it is more complete and well-written.”
Example of an acceptable one: “Response A states that [X], which is factually incorrect: [reasoning or source]. Response B, while somewhat longer than necessary, contains no factual errors and respects the informal tone constraint specified in the prompt.”
Pay and Time Management
The effective hourly rate on Aether falls between $20 and $35/hr depending on task complexity and your internal Quality Score.
Average time per task type:
| Task type | Estimated average time |
|---|---|
| General conversation | 8–14 minutes |
| Logical reasoning | 12–20 minutes |
| Light coding | 15–25 minutes |
| Complex instruction adherence | 10–18 minutes |
Coding tasks tend to take more time but are also compensated at a slightly higher rate. Don’t try to optimize for speed: Aether rewards accuracy, not throughput. A Quality Score that drops below threshold leads to suspension from the project, often without warning.
Productivity bonuses — referenced in official project communications — are awarded to annotators who maintain a high Quality Score and consistent task volume over a given period. They’re not guaranteed and vary cycle to cycle.
Who Should Apply
Aether isn’t suited to someone looking for completely passive work or who wants to “do something while watching TV.” The level of attention required for each individual task is high.
The ideal profile for Aether:
- Strong analytical logic: you can deconstruct an argument, identify where a causal chain breaks, recognize an unsupported claim.
- Obsessive attention to detail: you read prompts in full, including secondary clauses. You take nothing for granted.
- Native or near-native English: the project primarily involves English-language content and guidelines. Difficulty parsing technical English instructions is a concrete risk factor for rating errors.
- Comfort with argumentative writing: you can structure written reasoning clearly, without repetition or vagueness.
A formal academic background or prior machine learning experience is not required.
How to Get Assigned to Aether
Aether isn’t directly accessible when you first sign up on Outlier. Assignment happens through an internal qualification process covered in detail in a dedicated guide.
In short: after passing Outlier’s general screening, you can apply to specific projects or be automatically invited based on your profile and quality history. Aether tends to select annotators who are already active on the platform with a positive track record — it’s rarely offered to new users with no prior activity.
If you’re evaluating Outlier specifically to reach Aether, the right strategy is to first build a solid reputation on entry-level projects, maintain a high Quality Score, and from there apply or wait for an invitation.
What We Like
- One of the most consistent and reliable task flows on the platform
- Well-structured guidelines with dedicated support channels
- Productivity bonuses available for high-accuracy annotators
Areas for Improvement
- Rigorous admission assessment with deliberate trick questions
- Quality Score metrics monitored continuously
- Project instructions subject to frequent updates from the client
Ready to start earning?
Apply through our verified link to jump straight into the assessment and begin onboarding.
By clicking this link, you support our site at no extra cost.
Apply Now