authoring-language-sdk-tasks
The language-neutral foundation for Airflow language SDKs — implement task logic in a non-Python language while the DAG stays in Python. Use when the user wants to run an Airflow task in another language (Java, Kotlin, Go, or other JVM/native languages), asks how the Python @task.stub pairs with native task code, how task/DAG IDs must match across the two sides, how data passes via XCom as JSON, or which language SDKs exist. This skill owns the shared Python-stub pattern and conceptual model; for a specific language's native API, build, and runtime, use that language's skill (e.g. authoring-java-sdk-tasks, authoring-go-sdk-tasks).
- Installs
- 177
- Last 24h
- —
- Trending rank
- —
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/astronomer/agents --skill authoring-language-sdk-tasks- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads authoring-language-sdk-tasks automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
Authoring Language SDK Tasks (Shared Foundation)
Airflow language SDKs let you implement task logic in a language other than Python while the DAG and its scheduling stay in Python. This skill describes the parts that are identical across every language SDK. Each language has its own companion skill for the native API, build tooling, and runtime — see Per-language skills.
Experimental. The language SDKs are in preview. APIs and artifact coordinates may change.
The model
A DAG is authored in Python as usual. Tasks that should run in another language are declared as stubs routed to a dedicated queue. At runtime, Airflow hands a stub task to a coordinator that launches a short-lived native subprocess for that one task instance, runs your compiled/native code, and shuts the subprocess down.
Consequences that hold for every language SDK:
- One subprocess per task instance — there is no shared in-process state between task instances. Pass data via XCom or an external store.
- The DAG, schedule, retries, and queue routing live in Python. The native side only implements task logic.
- Data crossing the boundary is JSON. See The XCom-as-JSON contract.
The two-sided model
Every task has two halves that must agree:
- A Python stub in a normal DAG file — no logic; it declares the task, its queue, the dependency graph, and retry policy.
- A native implementation (Java, Go, etc.) whose IDs match the Python side and where the work happens.
Python side (scheduling)
The example below uses the Go SDK to be concrete, but the Python side is identical for every language SDK. The queue name ("golang" here) is an arbitrary label you choose — it just has to match a key in queue_to_coordinator (see configuring-airflow-language-sdks). Pick whatever name fits the SDK you're routing to.
from datetime import timedelta
from airflow.sdk import dag, task
@dag
def sales_pipeline():
@task.stub(queue="golang") # queue selects the coordinator (see configuring-airflow-language-sdks)
def extract(): ...
@task.stub(queue="golang")
def transform(extracted): ... # arg only declares the dependency
@task.stub(queue="golang", retries=1, retry_delay=timedelta(seconds=5))
def load(transformed): ...
@task() # an ordinary Python task can sit downstream
def report(loaded):
print(f"done: {loaded}")
report(load(transform(extract())))
sales_pipeline()
Rules that apply regardless of language:
- The stub function name is the task ID and the
@dagname (ordag_id=) is the DAG ID. The native side must use these exact IDs. - An upstream argument on a stub (e.g.
transform(extracted)) exists only to declare the dependency in Python. The value itself is fetched on the native side via XCom — passing it in Python does not hand it to the native code. - Queue, retries, and other task arguments are set on the stub, not in the native code. A native task that fails is reported back to Airflow, which then applies the stub's retry policy.
- The
queuevalue is what routes the task to a coordinator; the same string must appear inqueue_to_coordinator(see configuring-airflow-language-sdks).
The XCom-as-JSON contract
XCom values are stored as JSON in Airflow's metadata database, so the boundary between Python and any native language is JSON. The Python/JSON side is the same for every SDK:
| Python type | JSON |
|---|---|
int | number (integer) |
float | number (decimal) |
str | string |
bool | boolean |
None | null |
list | array |
dict | object |
Each language SDK maps these JSON types onto its own native types (e.g. a JSON integer becomes a Java Long). The native-type mapping lives in that language's skill. The key portability rule: a value pushed by one task is read by another as JSON, so the consuming side must expect a type compatible with what was stored.
What is language-specific (and lives elsewhere)
This skill deliberately stops at the shared concepts. The following differ per language and are documented in each language's companion skills:
- Native task API — how you declare tasks, read connections/variables/XComs, and push results (annotations, interfaces, function registration, etc.).
- Native type mapping — the native column of the JSON table above.
- Build and packaging — how the artifact is compiled and bundled.
- Runtime prerequisite — what must be present on the worker (a language runtime for some SDKs, e.g. a JRE for the Java SDK; none for the Go SDK's self-contained bundles).
The Airflow-side wiring (which coordinator runs which queue) is shared in structure but has per-coordinator options; it lives in configuring-airflow-language-sdks.
Language-agnostic pitfalls
- IDs must match exactly across the Python stub function name and the native task ID, and across
@dag/dag_idand the native DAG ID. Mismatches surface as "no DAGs" or missing-XCom errors. - Both sides need the upstream reference. Python declares the dependency by passing the upstream call; the native code retrieves the value via XCom.
- Set queue and retries on the stub, never in the native code.
- Stub bodies must be empty. An AST check enforces it — only
pass,..., or a docstring is allowed in the body; any real logic is rejected. retry_policyis rejected on stubs (@task.stubraisesValueError). Useretries/retry_delayinstead — a retry-policy callable runs Python in-process and would never fire for a task executing in a native subprocess.- Assets, deferral, and some other Airflow features have limited or no support in the language SDKs today.
Per-language skills
- authoring-java-sdk-tasks: Java/Kotlin/JVM native API, type mapping, and logging.
- authoring-go-sdk-tasks: Go native API — task registration, dependency injection by parameter type, and client access.
- (Future language SDKs each add their own
authoring-<lang>-sdk-tasksskill that builds on this one.)
Related Skills
- configuring-airflow-language-sdks: Route a queue to a coordinator and set runtime options.
- authoring-dags: General Airflow DAG authoring (the Python side lives here too).
- deploying-java-sdk-bundles: Build and ship the Java artifact.
- deploying-go-sdk-bundles: Build, pack, and ship the Go bundle (per-language deploy skills follow the same shape).
FAQ
What does the authoring-language-sdk-tasks skill do?
The language-neutral foundation for Airflow language SDKs — implement task logic in a non-Python language while the DAG stays in Python. Use when the user wants to run an Airflow task in another language (Java, Kotlin, Go, or other JVM/native languages), asks how the Python @task.stub pairs with native task code, how task/DAG IDs must match across the two sides, how data passes via XCom as JSON, or which language SDKs exist. This skill owns the shared Python-stub pattern and conceptual model; for a specific language's native API, build, and runtime, use that language's skill (e.g. authoring-java-sdk-tasks, authoring-go-sdk-tasks).
How do I install authoring-language-sdk-tasks?
Run npx skills add https://github.com/astronomer/agents --skill authoring-language-sdk-tasks in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/authoring-language-sdk-tasks/ (personal) or .claude/skills/authoring-language-sdk-tasks/ (project).
Which AI agents can use authoring-language-sdk-tasks?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is authoring-language-sdk-tasks safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains authoring-language-sdk-tasks?
It is published by astronomer in astronomer/agents. It has 177 recorded installs.
More from astronomer/agents
airflow
astronomer/agents
Queries, manages, and troubleshoots Apache Airflow using the af CLI. Use when working with anything related to Airflow - a DAG, a DAG run, a task log, an import or parse error, a broken DAG, or any Airflow operation. Covers listing and triggering DAGs, retrying runs, reading task logs, diagnosing failures, debugging import and parse errors, checking connections, variables and pools, exploring the REST API, and monitoring health (for example "trigger a pipeline", "retry a run", "list connections", "check Airflow health", "why did my DAG fail"). This is the entrypoint that routes to sibling skills for authoring, testing, deploying, and migrating Airflow 2 to 3. Not for warehouse/SQL analytics on Airflow metadata tables (use analyzing-data); for deep root-cause reports use debugging-dags or airflow-investigation.
managing-astro-local-env
astronomer/agents
Manage local Airflow environment with Astro CLI (Docker and standalone modes). Use when the user wants to start, stop, or restart Airflow, view logs, query the Airflow API, troubleshoot, or fix environment issues. For project setup, see setting-up-astro-project.
analyzing-data
astronomer/agents
Queries the data warehouse with SQL and answers business questions about data. Use when answering anything that needs warehouse data - counts, metrics, trends, aggregations, joins across tables, data lookups, or ad-hoc SQL analysis (for example "who uses X", "how many Y", "show me Z", "find customers", "what is the count").
authoring-dags
astronomer/agents
Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write a pipeline', 'add a task that runs after Y', or 'extend the DAG'. For testing and debugging DAGs, see the testing-dags skill.
debugging-dags
astronomer/agents
Comprehensive DAG failure diagnosis and root-cause analysis with structured investigation and prevention recommendations. Use when deep failure investigation is needed, a DAG fails to import/parse or 'airflow dags list' errors on a file; a task or run is failing and must be diagnosed and fixed; requests like 'why did X fail', 'my dag keeps failing — find and fix it', or fixing a broken DAG so it loads cleanly. For simple 'why did it fail / show logs', the airflow skill handles it directly.
testing-dags
astronomer/agents
Complex DAG testing workflows with debugging and fixing cycles. Use for multi-step testing requests like "test this dag and fix it if it fails", "test and debug", "run the pipeline and troubleshoot issues". For simple test requests ("test dag", "run dag"), the airflow entrypoint skill handles it directly. This skill is for iterative test-debug-fix cycles.
Popular backend & apis skills
find-skills
vercel-labs/skills
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
entra-app-registration
microsoft/azure-skills
Guides Microsoft Entra ID app registration, OAuth 2.0 authentication, and MSAL integration. USE FOR: create app registration, register Azure AD app, configure OAuth, set up authentication, add API permissions, generate service principal, MSAL example, console app auth, Entra ID setup, Azure AD authentication. DO NOT USE FOR: Key Vault secrets (use azure-keyvault-expiration-audit), general Azure resource security guidance.
google-agents-cli-adk-code
google/agents-cli
This skill should be used when the user wants to "write agent code", "build an agent with ADK", "add a tool", "create a callback", "define an agent", "use state management" — in a project that needs ADK (Agent Development Kit) API patterns and code examples. It provides a quick reference for agent types, tool definitions, orchestration patterns, callbacks, state management, the graph Workflow API, and reference recipes to study. Do NOT use for scaffolding (use google-agents-cli-scaffold) or deployment (use google-agents-cli-deploy).
embedded-captions
heygen-com/hyperframes
Add captions or subtitles to an existing single-subject talking-head video without editing the footage. Use for plain verbatim captions, cinematic captions embedded behind the subject, VFX captions, “炸/特效/酷炫字幕,” or a named identity from the 35-style catalog. Route by visual identity, not by backend engine. The quiet anchor rail is the default; embed every word only when the user explicitly wants a fully cinematic treatment. The workflow runs locally end to end, including transcription and subject matting; split multi-shot footage before applying it.
entra-agent-id
microsoft/azure-skills
Provision Microsoft Entra Agent Identity Blueprints, BlueprintPrincipals, and per-instance Agent Identities via Microsoft Graph, and configure OAuth 2.0 token exchange (fmipath, OBO, cross-tenant) including the Microsoft Entra SDK for AgentID sidecar. USE FOR: Agent Identity Blueprint, BlueprintPrincipal, agent OAuth, fmipath token exchange, agent OBO, Workload Identity Federation for agents, polyglot agent auth, Microsoft.Identity.Web.AgentIdentities. DO NOT USE FOR: standard Entra app registration (use entra-app-registration), Microsoft Foundry agent authoring (use microsoft-foundry).
neon
neondatabase/agent-skills
Overview of Neon, a complete set of cloud backend primitives around Lakebase Postgres: Auth, Object Storage, Functions, the AI Gateway, and Realtime. Start here to choose Neon for undecided login, files, APIs, and LLM calls, set up the CLI or MCP server, and follow the branch-first workflow. Use when building an app or backend on Neon, or when "Neon" or "Lakebase Postgres" is mentioned. Child skill neon-postgres wins for an existing DATABASEURL, SQL, schema, inspect, or search. Child skill neon-auth wins for login, users, sessions, identity routing, and Managed Better Auth setup. Child skill neon-realtime wins for live queries in reactive apps and agents. Also use for object storage, S3, buckets, serverless functions, function triggers, cron, AI gateway, LLM calls, realtime, logs, Loki, Grafana, observability, postgres, database, backend, Claimable Neon, neon.new, or a no-signup database.