This is the staging instance of ATR, for testing only. The production server is at releases.apache.org.

7.15. Input validation

Up: 7. Developer guide

Prev: 7.14. Authorization security

Next: 7.16. Dependency updates

Sections:

Overview

Input validation is critical for ATR's security posture. As a system that handles cryptographic signatures and release artifacts, ATR must ensure that all user input is properly validated before processing. This page documents the validation strategies and patterns used throughout the codebase.

Defense in depth

ATR employs multiple layers of validation:

  1. Transport layer: HTTPS required, enforced by httpd
  2. Request layer: ATR enforces the request body limit set by MAX_CONTENT_LENGTH
  3. Form layer: Pydantic models validate structure and types
  4. Application layer: Business logic validation in route handlers, interaction helpers, and storage writers
  5. Database layer: SQLAlchemy ORM with parameterized queries, plus constraints
  6. Markdown layer: Via cmarkgfm
  7. Output layer: Jinja2 auto-escaping for HTML output

Each layer provides independent protection, so a failure in one layer does not compromise the system.

Form validation with Pydantic

All form inputs in ATR are validated through Pydantic models defined in form.py. The base class for forms is Form (Form), which extends Pydantic's BaseModel.

Defining form fields

Form fields are defined using Python type annotations and the label (label) function:

class ExampleForm(Form):
    name: str = label("Project name", "Enter the project name")
    count: int = label("Count", widget=Widget.NUMBER)
    email: EmailStr = label("Contact email", widget=Widget.EMAIL)

The label function accepts a description (shown to users), optional documentation, and an optional widget hint for rendering.

Validation process

When a form is submitted, ATR:

  1. Extracts form data from the request via quart_request (quart_request)
  2. Passes the data to the Pydantic model for validation
  3. If validation fails, collects errors via flash_error_data (flash_error_data)
  4. Displays errors to the user with flash_error_summary (flash_error_summary)
  5. If validation succeeds, proceeds with the validated data

Pydantic provides built-in validators for common types (strings, integers, emails, URLs) and supports custom validators via decorators.

Custom validators

For complex validation logic, use Pydantic's @model_validator decorator:

from pydantic import model_validator

class ReleaseForm(Form):
    version: str = label("Version")

    @model_validator(mode="after")
    def validate_version_format(self):
        if not re.match(r"^\d+\.\d+\.\d+", self.version):
            raise ValueError("Version must start with X.Y.Z")
        return self

CSRF protection

All POST forms must include a CSRF token. The token is generated by csrf_input (csrf_input) and validated automatically by Quart-WTF:

def csrf_input() -> htm.VoidElement:
    csrf_token = utils.generate_csrf()
    return htpy.input(type="hidden", name="csrf_token", value=csrf_token)

In templates, include the CSRF token in every form:

<form method="post">
    {{ csrf_input() }}
    <!-- other form fields -->
</form>

The CSRF token is tied to the user's session and validated on form submission. Requests without a valid CSRF token are rejected. When using the form module renderer, the CSRF token is added automatically.

Validation rules by input type

ASF User IDs

User IDs are validated against a strict pattern in principal.py:

if not re.match(r"^[-_a-z0-9]+$", user):
    raise CommitterError("Invalid characters in User ID")

Only lowercase alphanumeric characters, hyphens, and underscores are permitted.

Email addresses

Email validation uses Pydantic's EmailStr type, which implements RFC 5322 validation:

from pydantic import EmailStr

class ContactForm(Form):
    email: EmailStr = label("Email address")

URLs

URL validation uses Pydantic's HttpUrl type:

from pydantic import HttpUrl

class LinkForm(Form):
    website: HttpUrl = label("Website URL")

Version strings

Version strings are validated according to project-specific patterns. The general pattern allows semantic versioning with optional suffixes:

VERSION_PATTERN = re.compile(r"^[0-9]+\.[0-9]+.*$")

Committee and project names

Committee and project names are validated against the set of known committees and projects from LDAP and the ASF project database. Unknown names are rejected.

File names

Release upload paths use safe.RelPath, which rejects absolute paths, directory traversal, null bytes and most dotfiles. These rules apply to every upload method, including rsync and SVN imports. For file content checks, see File handling.

Data integrity validation

Beyond input validation, ATR performs data integrity validation on database records using validate.py. This catches inconsistencies that may have been introduced by bugs, migrations, or manual database edits.

Committee validation

Committee records are not created through a form. They are synced from LDAP and Whimsy by _update_committees (_update_committees), so their fields have no Pydantic input layer, and the data integrity validators are the main place each field is confirmed.

The committee (committee) function checks:

  • key must be a valid committee key (the safe.CommitteeKey (CommitteeKey) character set)
  • name must be set, trimmed, and not prefixed with "Apache "
  • committee_members, committers, and release_managers entries must each look like an ASF UID
  • child_committees must be empty (not used)

Per-field coverage:

Field Input layer Data integrity
key none (LDAP project name) valid committee key
name none (Whimsy) set, trimmed, no "Apache " prefix
is_podling set during sync type-enforced (bool)
parent_committee_key none foreign key
committee_members none (LDAP) ASF UID format
committers none (LDAP) ASF UID format
release_managers none ASF UID format
child_committees none must be empty

Project validation

Projects do have an input layer. AddProjectForm (AddProjectForm) validates the key and display name at creation, and EditVersionSchemeForm (EditVersionSchemeForm) rejects a version_pattern or cycle_match that is not a valid regex (and a cycle_match with no capture group). Calver projects describe their cycles with a calver_format date format instead of a regex; see Calver cycle formats below. The data integrity validators mirror these and cover fields that have no form.

The project (project) function checks:

  • key must be a valid project key (the safe.ProjectKey (ProjectKey) character set)
  • category must use comma-separated labels without colons
  • committee_key must be set (project must be linked to a committee)
  • created timestamp must be in the past
  • created_by, if set, must look like an ASF UID
  • cycle_match, if set, must be a regex with at least one capture group
  • calver_format, if set, must be a valid date format with at least one calendar field
  • name must be set and start with "Apache "
  • programming_languages must use comma-separated labels without colons
  • version_pattern, if set, must be a compilable regex

Per-field coverage:

Field Input layer Data integrity
key safe.ProjectKey (charset, lowercase) + AddProjectForm (committee-prefixed) valid project key
name AddProjectForm and EditMetadataForm (Apache prefix, case rules); configuration API currently accepts names without the prefix set, "Apache " prefix
status none (enum) type-enforced (enum)
description EditMetadataForm (free text) none
category AddCategoryForm (free text) comma-separated, no colons
programming_languages AddLanguageForm (free text) comma-separated, no colons
version_method EditVersionSchemeForm (enum) type-enforced (enum)
version_pattern EditVersionSchemeForm (regex validity) compilable regex
cycle_match EditVersionSchemeForm (regex and capture group) regex with capture group
calver_format EditVersionSchemeForm (date format) valid format with a calendar field
branch_template EditVersionSchemeForm (free text) none, not currently enforced
committee_key safe.CommitteeKey must be set
created set at creation in the past
created_by none ASF UID format

Calver cycle formats

Calver projects declare the shape of their version strings with a date format string rather than a regex. The format is a sequence of tokens with literal separators between them:

Token Matches Role
YYYY / YY four- or two-digit year ordering
MM / M zero-padded or 1-2 digit month ordering
DD / D zero-padded or 1-2 digit day ordering
N a numeric serial (a patch ordinal within a calendar line) ordering

Wrap the part of the version that names the cycle in parentheses. A format with no parentheses keeps a single release track; one with a parenthesised span splits releases into one cycle per matched value. Trailing tokens are optional, so one format covers mixed-granularity histories - YYYYMMDD matches both 20130710 and 201407, reading the missing day as zero.

Examples:

Version Format Cycle Notes
2020.6.24 YYYY.M.D default non-padded month and day
2025-11-03 YYYY-MM-DD default dashed date
20130710 YYYYMMDD default run-together, day optional
09.04.01 (YY.MM).N 09.04 the YY.MM line is the cycle, N the patch

Release validation

The release (release) function checks:

  • created timestamp must be in the past
  • name must match the expected pattern for project and version
  • A draft's latest revision directory must exist, and a candidate's or preview's must not be empty
  • A released version must have no unfinished directory, except while its finalisation task is queued or active
  • package_managers must be empty (not used)
  • released timestamp must be in the past or None
  • sboms must be empty (not used)
  • Vote logic must be consistent (cannot have vote_resolved without vote_started)

Stored consistency checks

These checks find stored records that disagree with each other, and stored settings that ATR cannot use. Some write paths do not enforce these rules, as noted below. The checks report problems without blocking any action.

Rule Validator Source
A draft has no vote mode, resolution time, podling thread, voted revision, or vote thread URL. A failed or cancelled vote may leave its start time and vote serial number behind. release_vote_fields _resolve_transition
A candidate has a vote start time and mode, but no resolution time or vote thread URL. A preview has a start time and a mode, and also a resolution time if it has a vote serial number. A release catalogued from the distribution area may have no vote fields at all. release_vote_fields start_vote_no_commit, _resolve_transition
A vote serial number requires a vote start time. A voted revision requires a vote serial number, a start time, and a mode. Releases that voted before ATR recorded vote serial numbers or voted revisions may lack them. release_vote_prerequisites start_vote_no_commit
A candidate's or preview's voted revision, when recorded, is still its latest revision. release_voted_revision start_vote_no_commit
A draft, candidate, or preview belongs to an active project. Admins can retire a project that has such releases, which leaves them unable to progress. release_project_active ensure_project_active
An archive date or archive source requires the release to be archived. Older archived releases may have neither. This check is temporarily disabled. release_archived archive_release_core
A release's cycle is the one its project's current cycle matcher gives for its version, or the default cycle if matching fails. release_cycle reassign_release_cycles
A calver format is set only on calver projects, is valid, and generates the project's stored cycle matcher. project_calver_format edit_version_scheme, _apply_version_scheme
Release and policy download path suffixes are valid paths after filling in their template variables. An unset release suffix uses the policy suffix, and an empty one means the distribution root. Both are valid. release_download_path_suffix, project_download_path_suffix validate_download_path_suffix
Recipient defaults have only vote and announce entries. Each has a to string and cc and bcc lists of strings, and every address is on apache.org or one of its subdomains. The web forms and policy API do not check these addresses when saving them. An entry with an empty to and empty lists is allowed. project_recipient_defaults _set_recipient_defaults, _validate_recipient
The minimum voting period is unset, 0, or 72 to 168 hours. Template and RAT excludes URLs use HTTPS on an allowed host. A security contact, if set, is security@apache.org or the committee's own security@ address. project_min_hours, project_fetch_urls, project_security_contact validation.py
A Trusted Publishing repository name, if set, has no slash, and every workflow path starts with .github/workflows/. project_trusted_publishing validate_trusted_publishing_constraints
Each release finalisation task has a completed SVN publication task for the same project, version, and revision. That task has a valid stored result that records the SVN revision. This only audits the task records. Finalisation itself verifies the published files. release_finalise_publication release
A pending quarantine has a queued or active validation task. Without one, the upload stays pending and blocks the vote. quarantine_pending create_revision_with_quarantine
A staging or pending quarantine has no completion time and belongs to a draft. A failed or acknowledged quarantine has a completion time. A recorded prior revision belongs to the same release, even if that revision no longer exists. quarantine_completed, quarantine_release, quarantine_revision in quarantines create_revision_with_quarantine, quarantine.py
Each release's revision counter is at least its highest revision sequence number. For a draft, candidate, or preview, the vote counter is at least the highest vote serial number on the release, its ballots, or its vote initiation tasks. Gaps are allowed, and so are counters that outlive their release. allocation_counters populate_revision_sequence_and_key, start_vote_no_commit
An artifact's release key, if set, matches its project and version. A recorded signer or signature digest requires a signature path, and a digest is 64 lowercase hexadecimal digits. artifact_release_key, artifact_signature in artifacts release, catalogue_release
A system token has no owner, and an ordinary token's owner is its creator. personal_access_token_owner in personal_access_tokens tokens.py
Check result ignore patterns compile. check_result_ignores validate_ignore_pattern

Some rules depend on who is acting or on the moment of the request, so stored records cannot show whether they held. ATR enforces these only when the action happens:

  • Starting a vote or announcing a release proceeds only if the phase and latest revision still match the request. Resolving a vote checks the phase and vote serial number in the same way. Publishing rechecks the latest revision under a write lock. A stale request cannot move a release on.
  • A vote can start only when no checks are queued or running, no upload is waiting in quarantine, there are no blockers, the draft has at least one file, and the required concern groups are acknowledged. See Vote initiation.
  • The permitted vote and announce recipients depend on the acting user, the committee and its mailing lists, the project's stored defaults, the ATR_STATUS setting, and the podling vote round. See permitted_voting_recipients and permitted_announce_recipients in util.py.
  • An announcement requires a completed SVN publication of the exact preview revision, a recorded distribution for every file tag mapping, and an unchanged subject template. ATR also checks that the published files are available from every download server. If a server cannot be checked, the release manager can acknowledge this and continue, but missing or blocked files always stop the announcement. See the announcement writer.
  • Only the .asf.yaml route requires vote recipients on the committee's own apache.org subdomain. See ProjectConfigArgs.
  • The .asf.yaml route refuses a cycle matcher for a calver project, because ATR generates it from the calver format. See _apply_version_scheme.
  • A release version must match the project's version pattern when the release starts. Later pattern changes do not affect existing releases. See _validate_version.

Running validation

Data integrity validation can be run via the admin interface or programmatically:

async for divergence in validate.everything(data):
    print(f"{divergence.source}: {divergence.divergence}")

These validators are complementary to the live checks described below. Data integrity validation inspects stored records for drift or corruption. Business logic validation stops inconsistent actions before ATR accepts them.

Business logic validation

Field validation is only the first step. ATR also checks whether an action still makes sense in the wider state of the release. These rules compare data across releases, revisions, committees, queued tasks, stored policy, and message delivery settings. They live mainly in interaction.py, storage writers, mail.py, and shared helpers in util.py.

Vote initiation

Before a vote can start, release_ready_to_start_vote (release_ready_to_start_vote) checks that the release still has a latest revision and that the release is still attached to a committee.

That same validation step then checks the surrounding release state. The user must be a release manager for the project, meaning a PMC member or a designated committer release manager, or an ATR administrator. The latest revision must have no blocker results, and the release candidate draft must contain files. When ATR actually promotes the release into the voting phase, start_vote_no_commit (start_vote_no_commit) adds a task state check and refuses the transition while queued or active tasks still exist for that revision.

The same writer atomically enforces that the transition can only happen from the expected latest revision, using a caller supplied expected revision and a latest revision subquery in the UPDATE statement as a compare-and-swap guard. This binds vote initiation to release phase, revision state, policy, committee membership, check results, file storage, and task execution rather than to form fields alone.

Trusted Publishing

Trusted Publishing settings are validated when they are stored and again when they are used. On write, validate_trusted_publishing_constraints (validate_trusted_publishing_constraints) and policy.py normalize the configured repository name, branch, and workflow paths. The repository name is optional and is stored without a slash. Every workflow path must begin with .github/workflows/.

At request time, _trusted_project_checks (_trusted_project_checks) and _trusted_project (_trusted_project) compare the GitHub token claims with the stored policy. The repository must be under apache. The workflow reference must begin with that same repository, must include a git ref, and must resolve to a workflow path under .github/workflows/. ATR then looks up the project by the phase specific workflow path that was stored for compose, vote, or finish, and by repository: the one named in the policy when it sets one, or otherwise any GitHub repository in the project metadata. Distribution callbacks add one more contextual check in trusted_jwt_for_dist (trusted_jwt_for_dist), which refuses the request unless the named release exists and is in the expected phase for the requested operation. The cryptographic validation of the token itself is described in authentication security.

Email delivery

Email validation in ATR also depends on context. validate_email_recipients (validate_email_recipients) requires a primary recipient and rejects duplicate addresses across To, Cc, and Bcc. send (send) then requires the sender to use @apache.org, and _validate_recipient (_validate_recipient) rejects any envelope recipient outside @apache.org and its subdomains. This means that vote and release mail must go to ASF controlled addresses even if the address itself would be syntactically valid.

Output encoding

ATR uses Jinja2 for templating with auto-escaping enabled by default. All variables rendered in templates are automatically HTML-escaped:

<!-- This is safe; user_input is escaped -->
<p>Hello, {{ user_input }}</p>

When HTML output is intentionally generated (e.g., via htpy), it must be explicitly marked safe using markupsafe.Markup:

import markupsafe
safe_html = markupsafe.Markup("<strong>Bold</strong>")

For Markdown rendering, ATR uses markupsafe.Markup(cmarkgfm.github_flavored_markdown_to_html(markdown_text)), which safely filters dangerous input before rendering.

Never mark user-controlled data as safe without proper sanitization.

File upload security

See File handling for upload limits, format checks, archive quarantine and downloads.

Injection prevention

SQL injection

ATR uses SQLAlchemy ORM exclusively for database access. All queries use parameterized statements:

# Safe: parameterized query
result = await session.exec(
    select(Project).where(Project.key == project_name)
)

Direct SQL string concatenation is never used.

Cross-site scripting (XSS)

XSS is prevented through:

  • Jinja2 auto-escaping (enabled by default)
  • markupsafe.Markup for trusted HTML only
  • Content Security Policy headers set by ATR in server.py for application responses

Path traversal

Path traversal is prevented by:

  • Using pathlib.Path for all file operations
  • Validating that paths remain within expected directories
  • Rejecting file names containing path separators

For form fields that accept file or directory paths, always use form.RelPath (or form.RelPathList for multiple paths). These types automatically call to_relpath() (to_relpath), which rejects path traversal sequences, absolute paths, and empty values at the Pydantic validation layer. This is the preferred approach because it prevents path traversal before the handler code runs.

For cases outside of form validation (e.g., URL route parameters), use form.to_relpath() (to_relpath) directly, or validate manually:

import pathlib

base = pathlib.Path("/allowed/directory")
user_path = base / user_filename
# Verify the resolved path is still under base
if not user_path.resolve().is_relative_to(base.resolve()):
    raise ValueError("Path traversal detected")

Command injection

ATR safeguards against command injection as much as possible. Where external commands are necessary (e.g., GPG operations), arguments are passed as lists, never as shell strings:

import subprocess

# Safe: arguments as list
subprocess.run(["gpg", "--verify", signature_file, data_file])

# Unsafe: never do this
subprocess.run(f"gpg --verify {signature_file} {data_file}", shell=True)

Implementation references