authoring-java-sdk-tasks
Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. Use when the user wants to implement Airflow tasks in Java/JVM, asks about @Builder.Dag/@Builder.Task/@Builder.XCom, the Task/BundleBuilder interfaces, reading connections/variables/XComs from Java, the JSON-to-Java type mapping, or logging from Java tasks. This skill covers the Java-specific native API; the shared Python-stub pattern and conceptual model live in authoring-language-sdk-tasks. For building/shipping the bundle see deploying-java-sdk-bundles; for coordinator config see configuring-airflow-language-sdks.
- Installs
- 159
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/astronomer/agents --skill authoring-java-sdk-tasks- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads authoring-java-sdk-tasks automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Authoring Java SDK Tasks
The Airflow Java SDK implements the language-SDK model for the JVM: your DAG stays in Python, and each task instance runs in a short-lived JVM subprocess. This skill covers the Java-specific native API. The shared model — the Python @task.stub pattern, ID matching, and the XCom-as-JSON contract — lives in authoring-language-sdk-tasks; read that first if you're new to language SDKs.
Experimental. The Java SDK is in preview. Artifact coordinates and APIs may change.
Related skills: authoring-language-sdk-tasks (shared Python stub + concepts), configuring-airflow-language-sdks (route the queue to
JavaCoordinator), deploying-java-sdk-bundles (compile and ship the JAR).
Recap: the Python side
Java tasks are paired with Python stubs that carry no logic — they declare the task, queue, dependency graph, and retries. IDs must match the Java annotations exactly, and an upstream argument on a stub only declares the dependency (the value is fetched in Java). Full rules are in authoring-language-sdk-tasks; the minimal shape:
from airflow.sdk import dag, task
@dag
def sales_pipeline(): # dag_id "sales_pipeline" -> @Builder.Dag(id="sales_pipeline")
@task.stub(queue="java")
def extract(): ... # task_id "extract" -> @Builder.Task(id="extract")
@task.stub(queue="java")
def transform(extracted): ...
transform(extract())
sales_pipeline()
Java side: two APIs
Both APIs produce identical runtime behavior; pick by style, and you can mix them in one bundle.
Annotation-based API (recommended)
Annotate a plain class; an annotation processor generates the wiring (<ClassName>Builder) at compile time.
import static java.lang.System.Logger.Level.INFO;
import org.apache.airflow.sdk.*;
@Builder.Dag(id = "sales_pipeline") // must match the Python dag_id
public class SalesPipeline {
private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());
@Builder.Task(id = "extract") // must match the Python @task.stub name
public long extract(Client client) {
var conn = client.getConnection("sales_db");
log.log(INFO, "connected to {0}", conn.host);
return 42L; // return value is pushed as the return_value XCom
}
@Builder.Task(id = "transform")
public long transform(
Client client,
@Builder.XCom(task = "extract") long recordCount) { // pulls extract's return_value
var threshold = (String) client.getVariable("transform_threshold");
return recordCount * 2;
}
@Builder.Task // id omitted -> the method name "load" is used
public void load(Context context, @Builder.XCom(task = "transform") long transformed) {
log.log(INFO, "attempt {0}, value {1}", context.ti.tryNumber, transformed);
}
}
Annotation reference:
| Annotation | Purpose |
|---|---|
@Builder.Dag(id = "...") | Marks the class as a task container. id must match the Python dag_id; if omitted, the class name is used. Optional to = "..." renames the generated builder (default <ClassName>Builder). |
@Builder.Task(id = "...") | Marks a method as a task. id must match the Python @task.stub function name; if omitted, the method name is used. |
@Builder.XCom(task = "...", key = "...") | Injects an upstream task's XCom as a parameter. task defaults to the parameter name; key defaults to the producing task's return_value. The parameter type must be compatible with the stored JSON value. |
A task method's return value is automatically pushed as that task's return_value XCom. A method may declare throws Exception; any uncaught exception fails the task instance (which triggers retries if the stub configured them).
Interface-based API
Implement Task directly when you want full control over registration and XCom handling.
import org.apache.airflow.sdk.*;
public class ExtractTask implements Task {
@Override
public void execute(Context context, Client client) throws Exception {
var conn = client.getConnection("sales_db");
// ... do work ...
client.setXCom(42L); // push return_value explicitly
}
}
Register tasks manually in a Dag and expose it through a BundleBuilder:
public class MyBundle implements BundleBuilder {
@Override
public Iterable<Dag> getDags() {
var dag = new Dag("sales_pipeline"); // DAG ID matches Python
dag.addTask("extract", ExtractTask.class);
dag.addTask("transform", TransformTask.class);
return java.util.List.of(dag);
}
}
Each Task class needs a public no-arg constructor. Task IDs must be unique within a DAG, and DAG IDs unique within a bundle.
The entry point
Every bundle has a main that hands your DAGs to the SDK server. The server connects to the coordinator, runs one task instance, and exits.
import java.util.List;
import org.apache.airflow.sdk.*;
public class Main implements BundleBuilder {
@Override
public Iterable<Dag> getDags() {
// With the annotation API, the *Builder classes are generated at compile time.
return List.of(SalesPipelineBuilder.build());
}
public static void main(String[] args) {
Server.create(args).serve(new Main().build());
}
}
Server.create(args) parses the connection details Airflow passes on the command line — don't construct them by hand. Record this main class as the bundle's main class when you build it (see deploying-java-sdk-bundles).
Talking to Airflow from a task: Client
A Client is passed into every task and is scoped to the current DAG run and task instance.
| Call | Returns | Notes |
|---|---|---|
client.getConnection(id) | Connection | Fields: id, type, host, schema, login, password, port, extra. Any unset field is null. Throws if the connection doesn't exist. |
client.getVariable(key) | Object (or null) | Cast to the type you expect, e.g. (String) client.getVariable("threshold"). |
client.getXCom(taskId) | Object (or null) | Reads another task's return_value by default. Overloads accept key, dagId, runId, mapIndex, and includePriorDates for cross-DAG/run reads and mapped tasks. |
client.setXCom(value) | — | Pushes the return_value XCom (interface API). Value must be JSON-serializable. With the annotation API, returning a value does this for you. |
Context
The Context parameter exposes run metadata: context.dagRun (dagId, runId) and context.ti (dagId, runId, taskId, mapIndex, tryNumber). tryNumber is useful for retry-aware logic.
XCom: Java types
XComs cross the boundary as JSON (the shared contract is in authoring-language-sdk-tasks). When you read one back in Java you get:
| Python type | JSON | Java type from getXCom |
|---|---|---|
int | integer | Long (or BigInteger if too large) |
float | decimal | Double |
str | string | String |
bool | boolean | Boolean |
None | null | null |
list | array | List<Object> |
dict | object | Map<String, Object> |
Declare @Builder.XCom parameter types to match. A mismatch (e.g. declaring int when the value is a String) fails the task.
Logging
Declare a logger as a static field named after the class — the conventional pattern regardless of framework:
private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());
For records to reach Airflow's task log store (and show in the UI), the bundle must include one of the SDK logging integration artifacts (airflow-sdk-jpl, airflow-sdk-slf4j, airflow-sdk-log4j2, or airflow-sdk-jul). The dependencies and per-framework setup are in the logging integration section of deploying-java-sdk-bundles. System.Logger (JPL) with airflow-sdk-jpl is the lightest option and needs no configuration.
A complete worked example ships with the SDK
The SDK repository includes a runnable example under java-sdk/example/:
src/resources/dags/java_examples.py— Python DAGs pairing Python tasks with Java stubs, including aloadstub withretries=1.src/java/.../AnnotationExample.java— annotation API, including a task that fails ontryNumber == 1and succeeds on retry.src/java/.../InterfaceExampleBuilder.java— the same tasks via theTaskinterface andDag.addTask(...).src/java/.../ExampleBundleBuilder.java— aBundleBuilderreturning both DAGs plus themainentry point.
Point users there for an end-to-end reference.
Java-specific pitfalls
- Cast
Objectreturns deliberately.getVariableandgetXComreturnObject; match the cast to the JSON type (see the table above). @Builder.XComparameter types must match the stored JSON type, or the task fails at runtime.- The annotation processor must be on the build for the annotation API (generates
<ClassName>Builder); it is not needed for the interface API. See deploying-java-sdk-bundles. - See authoring-language-sdk-tasks for the language-agnostic pitfalls (ID matching, one JVM per task instance, queue/retries on the stub).
Related Skills
- authoring-language-sdk-tasks: Shared Python-stub pattern and concepts (read first).
- configuring-airflow-language-sdks: Route the
javaqueue toJavaCoordinatorand set JRE/coordinator options. - deploying-java-sdk-bundles: Build the bundle (Gradle/Maven) and place the JAR where Airflow can find it.
- authoring-dags: General Airflow DAG authoring.
FAQ
What does the authoring-java-sdk-tasks skill do?
Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. Use when the user wants to implement Airflow tasks in Java/JVM, asks about @Builder.Dag/@Builder.Task/@Builder.XCom, the Task/BundleBuilder interfaces, reading connections/variables/XComs from Java, the JSON-to-Java type mapping, or logging from Java tasks. This skill covers the Java-specific native API; the shared Python-stub pattern and conceptual model live in authoring-language-sdk-tasks. For building/shipping the bundle see deploying-java-sdk-bundles; for coordinator config see configuring-airflow-language-sdks.
How do I install authoring-java-sdk-tasks?
Run npx skills add https://github.com/astronomer/agents --skill authoring-java-sdk-tasks in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/authoring-java-sdk-tasks/ (personal) or .claude/skills/authoring-java-sdk-tasks/ (project).
Which AI agents can use authoring-java-sdk-tasks?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is authoring-java-sdk-tasks safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains authoring-java-sdk-tasks?
It is published by astronomer in astronomer/agents. It has 159 recorded installs.
More from astronomer/agents
airflow
astronomer/agents
Queries, manages, and troubleshoots Apache Airflow using the af CLI. Use when working with anything related to Airflow - a DAG, a DAG run, a task log, an import or parse error, a broken DAG, or any Airflow operation. Covers listing and triggering DAGs, retrying runs, reading task logs, diagnosing failures, debugging import and parse errors, checking connections, variables and pools, exploring the REST API, and monitoring health (for example "trigger a pipeline", "retry a run", "list connections", "check Airflow health", "why did my DAG fail"). This is the entrypoint that routes to sibling skills for authoring, testing, deploying, and migrating Airflow 2 to 3. Not for warehouse/SQL analytics on Airflow metadata tables (use analyzing-data); for deep root-cause reports use debugging-dags or airflow-investigation.
managing-astro-local-env
astronomer/agents
Manage local Airflow environment with Astro CLI (Docker and standalone modes). Use when the user wants to start, stop, or restart Airflow, view logs, query the Airflow API, troubleshoot, or fix environment issues. For project setup, see setting-up-astro-project.
analyzing-data
astronomer/agents
Queries the data warehouse with SQL and answers business questions about data. Use when answering anything that needs warehouse data - counts, metrics, trends, aggregations, joins across tables, data lookups, or ad-hoc SQL analysis (for example "who uses X", "how many Y", "show me Z", "find customers", "what is the count").
authoring-dags
astronomer/agents
Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a task that runs after Y', or 'extend the DAG'. For testing and debugging DAGs, see the testing-dags skill.
debugging-dags
astronomer/agents
Comprehensive DAG failure diagnosis and root-cause analysis with structured investigation and prevention recommendations. Use when deep failure investigation is needed, a DAG fails to import/parse or 'airflow dags list' errors on a file; a task or run is failing and must be diagnosed and fixed; requests like 'why did X fail', 'my dag keeps failing — find and fix it', or fixing a broken DAG so it loads cleanly. For simple 'why did it fail / show logs', the airflow skill handles it directly.
testing-dags
astronomer/agents
Complex DAG testing workflows with debugging and fixing cycles. Use for multi-step testing requests like "test this dag and fix it if it fails", "test and debug", "run the pipeline and troubleshoot issues". For simple test requests ("test dag", "run dag"), the airflow entrypoint skill handles it directly. This skill is for iterative test-debug-fix cycles.
Popular backend & apis skills
find-skills
vercel-labs/skills
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
entra-app-registration
microsoft/azure-skills
Guides Microsoft Entra ID app registration, OAuth 2.0 authentication, and MSAL integration. USE FOR: create app registration, register Azure AD app, configure OAuth, set up authentication, add API permissions, generate service principal, MSAL example, console app auth, Entra ID setup, Azure AD authentication. DO NOT USE FOR: Key Vault secrets (use azure-keyvault-expiration-audit), general Azure resource security guidance.
google-agents-cli-adk-code
google/agents-cli
This skill should be used when the user wants to "write agent code", "build an agent with ADK", "add a tool", "create a callback", "define an agent", "use state management" — in a project that needs ADK (Agent Development Kit) API patterns and code examples. It provides a quick reference for agent types, tool definitions, orchestration patterns, callbacks, state management, the graph Workflow API, and reference recipes to study. Do NOT use for scaffolding (use google-agents-cli-scaffold) or deployment (use google-agents-cli-deploy).
embedded-captions
heygen-com/hyperframes
Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet anchor rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.
entra-agent-id
microsoft/azure-skills
Provision Microsoft Entra Agent Identity Blueprints, BlueprintPrincipals, and per-instance Agent Identities via Microsoft Graph, and configure OAuth 2.0 token exchange (fmipath, OBO, cross-tenant) including the Microsoft Entra SDK for AgentID sidecar. USE FOR: Agent Identity Blueprint, BlueprintPrincipal, agent OAuth, fmipath token exchange, agent OBO, Workload Identity Federation for agents, polyglot agent auth, Microsoft.Identity.Web.AgentIdentities. DO NOT USE FOR: standard Entra app registration (use entra-app-registration), Microsoft Foundry agent authoring (use microsoft-foundry).
neon
neondatabase/agent-skills
Overview of Neon, a complete set of cloud backend primitives around Lakebase Postgres: Auth, Object Storage, Functions, the AI Gateway, and Realtime. Start here to choose Neon for undecided login, files, APIs, and LLM calls, set up the CLI or MCP server, and follow the branch-first workflow. Use when building an app or backend on Neon, or when "Neon" or "Lakebase Postgres" is mentioned. Child skill neon-postgres wins for an existing DATABASEURL, SQL, schema, inspect, or search. Child skill neon-auth wins for login, users, sessions, identity routing, and Managed Better Auth setup. Child skill neon-realtime wins for live queries in reactive apps and agents. Also use for object storage, S3, buckets, serverless functions, function triggers, cron, AI gateway, LLM calls, realtime, logs, Loki, Grafana, observability, postgres, database, backend, Claimable Neon, neon.new, or a no-signup database.