{"data":{"version":"v1","base_url":"https://api.softon.dev/v1","datasets":[{"slug":"jobs","name":"Job Data API","status":"live","operations":[{"id":"list-jobs","verb":"GET","path":"/v1/jobs","summary":"List job postings","returns":"array","params":[{"name":"q","in":"query","type":"string","required":false,"desc":"Full-text match on title, company and description.","example":"golang"},{"name":"remote","in":"query","type":"boolean","required":false,"desc":"Restrict to remote-eligible postings.","example":"true"},{"name":"company","in":"query","type":"string","required":false,"desc":"Company slug or name."},{"name":"updated_after","in":"query","type":"timestamp","required":false,"desc":"Only rows whose `ingested_at` is later than this instant — everything that CHANGED since your last poll, new and re-seen alike. Pass the newest `ingested_at` you have stored. Also returns a posting whose only change was going inactive: retirement deliberately does not move `ingested_at`, so this matches `deactivated_at` as well and a mirror is told when a posting dies.","example":"2026-08-16T09:00:00Z"},{"name":"first_seen_after","in":"query","type":"timestamp","required":false,"desc":"Only postings this platform saw for the FIRST time after this instant. A refresh re-observes thousands of rows; across a day about **1,400** are genuinely new, which is roughly 15 requests — measured by walking a 24-hour window to exhaustion, not estimated. `first_seen_at` never moves once set, so it is safe as a watermark.","example":"2026-08-16T09:00:00Z"},{"name":"posted_after","in":"query","type":"date","required":false,"desc":"ISO-8601 date. Filters on posted_at -- the date the SOURCE published the posting, not the date we first saw it. Page by order=ingested_at for the latter.","example":"2026-08-01"},{"name":"location","in":"query","type":"string","required":false,"desc":"Substring match on `location`, case-insensitive. That field is free text at whatever precision the source publishes, so this matches a city inside \"Bakı, Azərbaycan\" -- and equally matches a country on the sources that publish only one. Not a place lookup: nothing is geocoded, and a spelling the source does not use will not match.","example":"Bakı"},{"name":"employment_type","in":"query","type":"string","required":false,"desc":"Exact match on `employment_type`, case-insensitive, and treating `-` and `_` as the same character. Two vocabularies are in this column: sources that derive the value from body text emit a schema.org token, and the rest pass through the employer's own label. The fold joins the pairs that differ only in punctuation — `FULL_TIME`/`Full-time`, `PART_TIME`/`Part-time`, `TEMPORARY`/`Temporary` — and it does NOT join `CONTRACTOR`/`Contract` or `INTERN`/`Internship`, which are different words. Query those separately; nothing here maps one vocabulary onto the other.","example":"Full-time"},{"name":"job_function","in":"query","type":"string","required":false,"desc":"Substring match on `job_function`, case-insensitive. Each source's own taxonomy, not normalised between them, so treat this as a search rather than an enum.","example":"Bank services"},{"name":"career_level","in":"query","type":"string","required":false,"desc":"Substring match on `career_level`, case-insensitive. The source's own label, not normalised.","example":"Professional"},{"name":"has_salary","in":"query","type":"boolean","required":false,"desc":"Only postings that publish pay (`true`), or only those that do not (`false`). Most do not: a minority of sources quote a figure at all, so this is usually the first filter to reach for if you are working with pay. Either end counts — a posting advertising a floor with no ceiling publishes pay.","example":"true"},{"name":"salary_from_at_least","in":"query","type":"number","required":false,"desc":"Minimum published pay, compared against `salary_from` — the FLOOR a posting quotes, not the ceiling. A posting offering 800–2000 does not pay at least 1500, and matching it on `salary_to` would answer a different question. Postings that publish no floor are excluded rather than assumed: null means the source did not say. Nothing is converted between currencies — pair this with `salary_currency`.","example":"1500"},{"name":"salary_currency","in":"query","type":"string","required":false,"desc":"Exact match on the source's own currency code, case-insensitive, usually AZN. Pay figures are not converted, so a bare number means different things on different postings; this is how you pin one.","example":"AZN"},{"name":"count","in":"query","type":"boolean","required":false,"desc":"Adds `meta.total` — how many rows the filter matches in ALL, across every page. `meta.count` is and remains the size of THIS page.\n\nOpt-in because it is a second query over the whole match, and a caller paging a feed should not pay for a number they never read. Exact rather than estimated: a filtered total is something you put on a page (\"1,247 jobs in Sumqayıt\"), and an estimate that is wrong by an unbounded amount is worse than no number.\n\nIf the count fails the page still returns, without `total` — the rows are the answer.","example":"true"},{"name":"include","in":"query","type":"string","required":false,"desc":"Opt a collection into fields that are off by default. One value today: `bodies`, which adds `description_html` and `requirements_html` to every row.\n\nThey are off by default because a body roughly doubles the size of a row that has one, and most callers walking 100 at a time do not want that. Ask for them when you do — hydrating a corpus one `/v1/jobs/{id}` call at a time would cost more requests than any plan allows, which made a documented feature unreachable.\n\nComma-separated for future values. An unrecognised token is a 400 rather than being ignored, so a typo cannot look like it worked.","example":"bodies"},{"name":"city","in":"query","type":"string","required":false,"desc":"Exact match on the normalised `city`, case-insensitive so the diacritics do not have to be typed. Not a substring search, deliberately: matching a city name loosely is what makes an employer called \"Bakı Bank\" in Sumqayıt show up under Bakı. Only postings whose source published a place are returned; ones whose location was recovered from the posting body are not, because that city may be the head office.","example":"Sumqayıt"},{"name":"district","in":"query","type":"string","required":false,"desc":"Exact match on the Baku rayon, case-insensitive. Narrower than `city` and only ever set for Baku postings whose source named a rayon.","example":"Nəsimi"},{"name":"country","in":"query","type":"string","required":false,"desc":"Exact match on the ISO 3166-1 alpha-2 `country`, case-insensitive — `?country=AZ`. This is the filter for \"only postings I can actually serve\": before it existed, an Azerbaijan-market board had no way to ask this API for Azerbaijan, and a source publishing its global board put Houston roles on two Azerbaijani job boards.\n\n**ISO 3166-1 alpha-2 — the two-letter code, not a name and not a language tag.** The United Kingdom is `GB`, never `UK`; `EN` is a language and not a country at all. One code, not a list: `?country=AZ,GE` and a repeated `?country=` do not select two countries. Any value that is not two letters is a `400` naming what you sent — except an empty `?country=`, which is the same as not sending it and applies no filter. `UK` gets a `400` telling you the code is `GB` — an empty page for a mistyped code reads as \"you hold nothing there\", which is the one thing this filter must never say by accident. A code the standard DOES define but this corpus has no rows for returns an empty page, because that answer is true.\n\n**It returns only postings whose source PUBLISHED a country**, which is a minority of the corpus — most rows are null because the board never said. So `?country=AZ` is much narrower than \"jobs in Azerbaijan\", and there is no way to ask for the nulls. If you want everything except the foreign rows, filter them out on your side by the country you get back rather than asking for one here.","example":"AZ"},{"name":"deadline_after","in":"query","type":"date","required":false,"desc":"ISO-8601 date. Only postings whose application deadline is on or after it — pass today's date for \"still open\". Postings with no deadline are excluded, and most have none, so this narrows hard. Unlike `posted_at` a future date is kept rather than nulled: that is what a deadline is.","example":"2026-09-01"},{"name":"active","in":"query","type":"string","required":false,"desc":"Which postings to return. `any` — **the default today** — is every posting this platform holds; `true` is only those whose source was still publishing them at its last complete scrape; `false` is only the retired ones. The default is `any` rather than `true` on purpose — a retired posting is accurate data with an archival use, unlike the inactive half of the quotes pool — but if you are showing postings to jobseekers, pass `true`.\n\n**This default is going to move to `true`**, and a default is part of the contract, so it is announced ahead of time under Queued on `/docs/shared/versioning` rather than shipped quietly. Passing `active` explicitly is immune to the change either way: `any` keeps today's behaviour permanently, `true` gets tomorrow's now.\n\n**Most of the corpus is retired**: 39,046 postings were still open of 149,383 held when this was measured (2026-08-28). So the default walk returns roughly four rows of history for every live vacancy, and coverage of the optional fields is markedly worse on the retired ones. If you are sizing the dataset rather than archiving it, pass `true` and read `live_records` on `GET /v1/sources`.","example":"true"},{"name":"source_type","in":"query","type":"string","required":false,"desc":"Restrict to whole CLASSES of publisher, rather than to named sources. Comma-separated; the one filter here that takes a list, because the question it answers is a union.\n\n`employer` — one organisation's own careers page, government bodies and universities included. `platform` — a multi-tenant careers platform, where each posting is still an employer's own (Glorri). `agency` — a recruitment or staffing firm posting for clients. `board` — a multi-employer job board. `aggregator` — republishes other boards' postings. `unknown` — four archive stems with no module and no registry entry, classified honestly rather than guessed.\n\nSo **everything that did not come from a job board** is `?source_type=employer,platform,agency` — one filter instead of fifty `?source=` calls. Measured 2026-08-28, that is 2,150 of the 39,046 live postings; the boards are 31,775 of them. Size it before you build on it.\n\n**This is our classification, not a fact the source asserts.** It is the one editorial field in this API: a value can change when a source is re-read, and the calls most likely to be argued with are recorded in the repository. The SET of values is stable. `GET /v1/sources` carries each source's `source_type` so you can see what a filter will include.","example":"employer,platform,agency"},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, `posted_at`, each optionally prefixed with `-` to reverse. Default is `-posted_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-posted_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\". Pass it to /v1/jobs/{id}."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper produced this posting. A stable slug, and half of `id` — this is what `?source=` takes, so it never changes even when a board renames itself. `GET /v1/sources` lists every slug with its display name, its dataset, how many rows it currently has and when it last landed a batch. That endpoint needs no key."},{"name":"source_name","type":"string","nullable":true,"desc":"The same source as a person would write it — `ABB` for `abb`, `1is.az` for `1is_az`. This is the one to display. Null for the handful of sources the scraper registry has no entry for, rather than repeating the slug."},{"name":"source_url","type":"string","nullable":true,"desc":"The source's own website — `https://glorri.com` for `glorri`, `https://abb-bank.az` for `abb`. This is the one to link `source_name` to.\n\n**Not this posting's link** — that is `url` — and not the source's listing page either. `GET /v1/sources` carries both, as `url` and `jobs_url`, and needs no key; `jobs_url` there is the page this platform reads the source's vacancies from, which is the better link if you are sending somebody to browse a source rather than to read about it.\n\nSet for 122 of 130 sources. Null on eight rather than guessed: three are hosted entirely on a third party's applicant-tracking system and their own domain appears nowhere in our scrapers (their `jobs_url` on `/v1/sources` still points at the ATS page the postings are actually on), one has only its ATS vendor's domain recorded against it, `other` is a catch-all bucket with no website, and **three we have measured as gone** — a hosting placeholder on every path, an organisation that no longer exists and now redirects to a different agency, and a decommissioned ATS tenant. Those three keep their rows and lose their link, because a link to nothing is worse than none."},{"name":"title","type":"string","nullable":false,"desc":"Posting title as published."},{"name":"company","type":"object","nullable":true,"desc":"The hiring company. `company.name` is always present when the object is; `company.slug` is nullable and is null on most rows -- do not type it as required. The whole object is null when the source omits the company, and is ALWAYS null on classifieds sources: there the advert is placed by a private individual rather than an employer, so there is no hiring company to name and publishing the poster under this field would be publishing a person's name as a company (#83)."},{"name":"location","type":"string","nullable":true,"desc":"Free text at whatever precision the source itself publishes — a city, a city and country, a district, an office or branch name, or the country alone. Not normalised, not geocoded, and not comparable between sources. Some boards publish no city at all and this is their country; nothing is inferred from the posting body to fill the gap, because a city read out of prose is as likely to be the employer's head office as the job's location."},{"name":"city","type":"string","nullable":true,"desc":"**25% of live postings** — 9,690 of 39,046, counted exactly on 2026-08-28 by summing `?city=` across the whole vocabulary. This said 60% and climbing, which was a projection rather than a measurement and is not where it is heading: the field fills only from sources that publish a structured place, about twenty of a hundred and thirty do, and several of the largest sources never will. Nine in ten of the values are `Bakı`. The same place as `location`, normalised to a closed vocabulary of Azerbaijani cities so it can be filtered and grouped — which `location` cannot be. **Null unless the source published a place of its own**: a city inferred from advert prose is as likely to be the employer's head office as the job's, so postings whose location was recovered that way carry a null city and keep their `location`. Azerbaijani cities only — a posting in Moscow or Warsaw is null here and unchanged in `location`. `location` is untouched and stays the higher-precision value; this is the comparable one."},{"name":"district","type":"string","nullable":true,"desc":"The Baku rayon, where the source names one — `Nəsimi`, `Binəqədi`, `Xətai` and the other nine. Null everywhere else, including for Baku postings that name no rayon. Implies `city`: a rayon is Baku's, so `city` is always `Bakı` when this is set. Present because one source publishes a rayon where every other publishes a city, and because a per-rayon page needs rayon granularity that a city vocabulary cannot express."},{"name":"country","type":"string","nullable":true,"desc":"The posting's country as **ISO 3166-1 alpha-2** — `AZ`, `PL`, `US`. **Null unless the source itself published a country**, which most do not: an Azerbaijani board's postings are Azerbaijani by construction and it never says so on the row. So null means *nobody said*, NOT *somewhere else* — use this to narrow what you serve, never to conclude a posting is foreign.\n\nIndependent of `city`, not a widening of it. `city` is an Azerbaijani-only vocabulary, so a Warsaw posting is null there and `PL` here, while a Baku posting whose source named no country is `Bakı` there and null here.\n\nIt comes from a structured country field where the source publishes one (Oracle ATS `PrimaryLocationCountry`, jobpilot's `country_code`) and from a country component of `location` where the source appends one — the same provenance rule as `city`: nothing is inferred from the posting body, and a token that is not an ISO code is stored as null rather than published."},{"name":"remote","type":"boolean","nullable":true,"desc":"Whether the posting is remote-eligible."},{"name":"employment_type","type":"string","nullable":true,"desc":"Full-time, Part-time, Contract, as the source labels it."},{"name":"job_function","type":"string","nullable":true,"desc":"The source's own function taxonomy, not normalised across sources."},{"name":"career_level","type":"string","nullable":true,"desc":"The source's own seniority label. Not normalised, so it is not called \"seniority\"."},{"name":"salary_from","type":"number","nullable":true,"desc":"Bottom of the published pay range, in `salary_currency`. Null where the source does not publish pay, which is most postings -- and null rather than 0, which would be a figure. Nothing is inferred from the posting body."},{"name":"salary_to","type":"number","nullable":true,"desc":"Top of the range. Either end can be null on its own: plenty of postings advertise a floor with no ceiling."},{"name":"salary_currency","type":"string","nullable":true,"desc":"The source's own currency code for the range, usually AZN. Not converted, so two postings in different currencies are not comparable by number alone."},{"name":"salary_period","type":"string","nullable":true,"desc":"What the figures are per: `HOUR`, `DAY`, `WEEK`, `MONTH` or `YEAR`. **Null unless the source itself stated it**, which today is about 1% of live postings and 6% of those that carry pay at all, and always `MONTH` — the eight sources that publish a period all publish that one. Null is the useful answer, not a gap: it means nobody knows, so do not assume. If you emit JobPosting `unitText`, omit it when this is null rather than defaulting to `MONTH`; a figure published under the wrong period is wrong by 12x. Nothing here is inferred from the amount, and nothing is read out of the posting body — a period word near a salary is usually about a shift, a rest day or required experience, not about pay."},{"name":"deadline_at","type":"timestamp","nullable":true,"desc":"Application deadline as published, RFC 3339. Unlike `posted_at` a future date is kept rather than nulled -- a date the clock has not reached is what a deadline is. Not a liveness signal: a posting is not withdrawn here when its deadline passes, and most postings have none."},{"name":"posted_at","type":"timestamp","nullable":true,"desc":"When the posting first appeared, RFC 3339."},{"name":"url","type":"string","nullable":true,"desc":"The posting's page on the source site — usually canonical, but see the last paragraph: on some sources it is a third-party redirect carrying search-session state rather than a canonical page. **Not unique**, in two different ways, and **not stable over time** in a third.\n\n**Same source.** A board that lists one vacancy under two of its own ids yields two rows that differ by `id` and match by `url`, and they can land in the SAME page of a collection — so a consumer keying a table on `url` must deduplicate before writing, or one duplicated pair will reject the whole statement and take the other 98 rows of the page with it. Collapse these by keeping the newer `ingested_at`.\n\n**Different sources.** An aggregator republishes another board's posting under its own id, carrying the ORIGINAL board's url. You then hold two records for one job with the same `url`, different `source`, and different `id`. **This pair cannot be collapsed the same way** — the two disagree about which `\u003csource\u003e:\u003csource_job_id\u003e` addresses the posting, so whichever you keep decides the id you can look it up by afterwards. Pick the record whose `source` owns the url's domain, or skip aggregators entirely: `GET /v1/sources` marks them with `\"aggregator\": true`.\n\n**Over time.** The same `id` can return a DIFFERENT `url` on any scrape, so a mirror cannot key on it — this is the arity the other two paragraphs do not cover, and unlike them it cannot be absorbed by deduplicating a page. Measured on `jobpilot_remote`: one id emitted **39 distinct urls over five weeks** for one unchanged posting. Both the path and the query rotate (`/away/` vs `/desc/`), and `jobAge` increments by exactly 24 every scrape, so the url is **guaranteed** to differ on every daily run. It is source-specific rather than fleet-wide: 11,722 of 11,722 sampled active urls on that source carry tracking parameters, against 0 of 100 on each of boss.az, tap.az, jobsearch.az and Glorri. Those redirect urls are also not canonical pages — `ckey=` is the search query somebody typed, not part of the posting's address.\n\nSo: **`id` is the identifier, and this is a link that moves.** Key on `id`, store `url` as an attribute, and if you must group by url, strip the query string first."},{"name":"description","type":"string","nullable":true,"desc":"Full posting body. Present when the crawler has fetched the posting's own page, null otherwise — on collections and on /v1/jobs/{id} alike, since neither fetches on demand.\n\n**92% of live postings**, counted over a complete walk of them on 2026-08-28, and 97% on `source_type=employer,platform,agency`. The gap is per-source and mostly upstream: a handful of sites give a listing and no reachable detail page. It is markedly worse on RETIRED postings, so a walk left at today's default `?active=any` will look thinner than the live data actually is."},{"name":"requirements","type":"string","nullable":true,"desc":"Requirements section when the source separates it, subject to the same condition as description. **Present on about 11% of live postings** (this said 5%, measured against a different denominator) — most boards write one prose block and it lands in `description`. Concentrated rather than spread: Glorri separates it on virtually every posting and boss.az on most, and the rest of the fleet barely at all. Do not build a layout that needs this."},{"name":"facts","type":"object","nullable":true,"desc":"The upstream columns this platform does not publish as their own field, under one key because only a handful of sources fill them. Five of the keys the live fleet fills — `work_type`, `company_logo`, `view_count`, `contact_email` and `contact_phone` — are typed fields of their own now and are still echoed here as strings; the typed fields are the ones to read, and `view_count` in particular can appear here as an abbreviated string like `1.3K` that will not cast. `slug` is the source's own. The two contact keys are rare — see `contact_email` for where each one comes from and for the two conditions an address named in a posting body has to meet before it is published. Note that a minority of rows imported from the upstream board's database carry an address THAT board extracted from advert text before the import, under rules this platform does not apply to its own crawling. They are the employer's own application contact for that posting, given for applications: do not use them to build a contact list, and do not send anything to them that is not a reply to the posting they came with. Postings imported from the upstream job board's own database — a large part of the archive — carry more, including that board's internal bookkeeping (`is_active`, `flag_status`, `created_at` and `click_count`). **Do not read those back.** They are the origin board's own columns about its own copy, frozen at the moment we imported its database — a mirror reading `facts.is_active` would resurrect its own retired rows from its own stale data. The `active` field is this platform's answer and the only one that moves. Treat the key set as per-row and per-source rather than as a schema: check for a key, do not assume it. Every value is rendered as a string whatever its underlying type, and a nested object or array is dropped rather than serialised. Unlike every other nullable field here the key is ABSENT rather than null on a posting with none of them. Salary and the deadline used to live here and are now typed fields of their own; they are still echoed inside `facts` as strings, and the typed columns are the ones to read."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored. Moves every time a scrape re-sees the posting."},{"name":"first_seen_at","type":"timestamp","nullable":false,"desc":"When this platform first saw the posting. Never changes — this is what distinguishes a new posting from one you already hold, since ingested_at moves on every re-sighting."},{"name":"times_seen","type":"integer","nullable":false,"desc":"How many scrapes have seen this posting. A source whose rows are all at 1 is changing its own identifiers rather than posting new work."},{"name":"active","type":"boolean","nullable":false,"desc":"**A verdict we can only give when the source has run.** `false` is a real finding: the source delivered a complete scrape, this posting was not in it, and nothing had seen it for 24 hours — roughly three consecutive misses on the published schedule.\n\n`true` is **weaker than it looks, and you should not read it as \"verified live today\"**. It is the default state, and the only thing that ever changes it is the retirement pass — which fires only when the source itself completes a run. A source that has stopped being scraped produces no verdict at all, so every one of its postings stays `true` indefinitely, and this description used to illustrate that with a number that was simply wrong — `jobsite_az` holding \"over 12,000 postings, all still `true`\". It holds 12,667 rows of which **1,199 are active**; the other 11,468 were retired on 2026-08-12, before that sentence was ever written. The figure was a corpus total relabelled as an active count.\n\n**You no longer have to take our word for the size of it.** `GET /v1/sources` publishes `live_records` beside `records` per source, so the gap between the two is exactly the population whose `true` carries no verdict — and a source whose `last_ingested_at` is old is one where that gap is unmeasured rather than small. Two sources report thousands of `records` and zero `live_records`.\n\n**If liveness matters to you, derive it yourself** — you have everything you need and it is four lines. Compare this posting's `ingested_at` against its source's `last_ingested_at` from `GET /v1/sources`: the source ran recently AND carried this row means live; the source ran recently and did NOT means gone; the source has not run means nobody knows, including us. See the worked example in the freshness docs. Nothing stops you being stricter than we are, and for a jobseeker-facing product you probably should be."},{"name":"deactivated_at","type":"timestamp","nullable":true,"desc":"When this platform concluded the posting had gone, RFC 3339, and null while it is active. Not when the employer took it down — nothing here can observe that — but at most one grace window later. `ingested_at` deliberately does not move when a posting is retired, so this is the timestamp that changed; `updated_after` matches on either."},{"name":"work_type","type":"string","nullable":true,"desc":"The source's own working-arrangement label — on-site, hybrid, shift — as it writes it. Not normalised, and not the same axis as `employment_type`, which is about the contract. Null on most postings."},{"name":"company_logo","type":"string","nullable":true,"desc":"Absolute URL of the company's logo on the source's own site. Not mirrored and not revalidated, so it can 404 independently of the posting — treat a broken image as expected rather than as an error."},{"name":"view_count","type":"integer","nullable":true,"desc":"How many times the SOURCE says the posting has been viewed — its counter, not ours, and not comparable between sources. Approximate where the source abbreviated it: a board showing 1.3K has published somewhere in 1,250–1,349, and that precision is gone before it reaches us. Null where the source publishes no counter."},{"name":"contact_email","type":"string","nullable":true,"desc":"**About 6% of live postings, and the distribution matters more than the number.** About 85% of these rows are on job boards, nearly all of that on four of them — jobsearch.az, boss.az, work.az and position.az — because a board's posting form asks the employer for a contact address. On `source_type=employer` it is about 28%, and on a board like boss.az it is every row. **If you filter to direct employers and also need an address, those two requirements pull against each other**: a company's own careers page routes applications through its own form or ATS, so most such postings publish no address at all and `url` is the apply path.\n\nWhere it is present it is the employer's own application mailbox for this posting, lower-cased, from one of two places and no others:\n\n1. a **structured field** the source published — an API field, a `mailto:` or a `tel:` link — read by nine of the ~120 scrapers;\n2. an **application mailbox named in the posting itself**, under three conditions that all have to hold: the **whole local part** is a role name from a closed list (`cv@`, `hr@`, `karyera@`, `vacancy@`, `tecrube@` …) with nothing appended, so `hr.aliyeva@` does not qualify; the **domain is the posting's own**; and the posting comes from a source whose url identifies **one organisation** — a direct employer or a named recruitment agency, never a job board, a multi-tenant careers platform or a hosted ATS domain shared between employers. Together those mean a named individual's address cannot qualify and neither can a third party's. Ten distinct mailboxes account for every row filled this way.\n\nNothing else is read out of advert prose, deliberately. Use it to reply to the posting it arrived with; do not use it to build a contact list."},{"name":"contact_phone","type":"string","nullable":true,"desc":"**About 4% of live postings**, and 98% of those are on jobsearch.az and boss.az alone — on employer career pages it is effectively absent (one row in the whole segment). Formatted exactly as the source published it, with no canonicalisation, because a number's formatting is theirs and normalising it here would be inventing one. Do not assume E.164.\n\nUnlike `contact_email` this comes ONLY from a structured field — an API field or a `tel:` link. Nothing is read from the posting body, and the asymmetry is deliberate: a number has no local part, so there is no way to tell an employer's application line from someone's mobile the way a role mailbox can be told from a person's. Across a complete walk of the live corpus there were no `tel:` links in any posting body, so there is nothing being withheld."}]},{"id":"get-job","verb":"GET","path":"/v1/jobs/{id}","summary":"Retrieve a job","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The posting id from a list response. Ids are stable and are never reissued, and this endpoint resolves a retired posting as readily as a live one — an id you already hold never starts returning 404. A posting is not deleted when its source removes it: `active` becomes false and `deactivated_at` is stamped, every other field keeps the last value that was scraped, and the row stays retrievable.\n\nRetrieving one by id is not the same as walking a collection. This endpoint resolves a retired posting whatever `?active=` would have done with it, and that does not change. What a COLLECTION contains does depend on the filter: rows withdrawn on request are the only ones that ever leave one regardless of how you ask, but once the announced `?active=` default moves to `true` a retired posting will be absent from a default walk while still resolving here. Pass `active=any` if you want a collection to keep matching this endpoint's reach.","example":"glorri:unibank-region-filiallari-uzre-konullu-seki-79999900"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\". Pass it to /v1/jobs/{id}."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper produced this posting. A stable slug, and half of `id` — this is what `?source=` takes, so it never changes even when a board renames itself. `GET /v1/sources` lists every slug with its display name, its dataset, how many rows it currently has and when it last landed a batch. That endpoint needs no key."},{"name":"source_name","type":"string","nullable":true,"desc":"The same source as a person would write it — `ABB` for `abb`, `1is.az` for `1is_az`. This is the one to display. Null for the handful of sources the scraper registry has no entry for, rather than repeating the slug."},{"name":"source_url","type":"string","nullable":true,"desc":"The source's own website — `https://glorri.com` for `glorri`, `https://abb-bank.az` for `abb`. This is the one to link `source_name` to.\n\n**Not this posting's link** — that is `url` — and not the source's listing page either. `GET /v1/sources` carries both, as `url` and `jobs_url`, and needs no key; `jobs_url` there is the page this platform reads the source's vacancies from, which is the better link if you are sending somebody to browse a source rather than to read about it.\n\nSet for 122 of 130 sources. Null on eight rather than guessed: three are hosted entirely on a third party's applicant-tracking system and their own domain appears nowhere in our scrapers (their `jobs_url` on `/v1/sources` still points at the ATS page the postings are actually on), one has only its ATS vendor's domain recorded against it, `other` is a catch-all bucket with no website, and **three we have measured as gone** — a hosting placeholder on every path, an organisation that no longer exists and now redirects to a different agency, and a decommissioned ATS tenant. Those three keep their rows and lose their link, because a link to nothing is worse than none."},{"name":"title","type":"string","nullable":false,"desc":"Posting title as published."},{"name":"company","type":"object","nullable":true,"desc":"The hiring company. `company.name` is always present when the object is; `company.slug` is nullable and is null on most rows -- do not type it as required. The whole object is null when the source omits the company, and is ALWAYS null on classifieds sources: there the advert is placed by a private individual rather than an employer, so there is no hiring company to name and publishing the poster under this field would be publishing a person's name as a company (#83)."},{"name":"location","type":"string","nullable":true,"desc":"Free text at whatever precision the source itself publishes — a city, a city and country, a district, an office or branch name, or the country alone. Not normalised, not geocoded, and not comparable between sources. Some boards publish no city at all and this is their country; nothing is inferred from the posting body to fill the gap, because a city read out of prose is as likely to be the employer's head office as the job's location."},{"name":"city","type":"string","nullable":true,"desc":"**25% of live postings** — 9,690 of 39,046, counted exactly on 2026-08-28 by summing `?city=` across the whole vocabulary. This said 60% and climbing, which was a projection rather than a measurement and is not where it is heading: the field fills only from sources that publish a structured place, about twenty of a hundred and thirty do, and several of the largest sources never will. Nine in ten of the values are `Bakı`. The same place as `location`, normalised to a closed vocabulary of Azerbaijani cities so it can be filtered and grouped — which `location` cannot be. **Null unless the source published a place of its own**: a city inferred from advert prose is as likely to be the employer's head office as the job's, so postings whose location was recovered that way carry a null city and keep their `location`. Azerbaijani cities only — a posting in Moscow or Warsaw is null here and unchanged in `location`. `location` is untouched and stays the higher-precision value; this is the comparable one."},{"name":"district","type":"string","nullable":true,"desc":"The Baku rayon, where the source names one — `Nəsimi`, `Binəqədi`, `Xətai` and the other nine. Null everywhere else, including for Baku postings that name no rayon. Implies `city`: a rayon is Baku's, so `city` is always `Bakı` when this is set. Present because one source publishes a rayon where every other publishes a city, and because a per-rayon page needs rayon granularity that a city vocabulary cannot express."},{"name":"country","type":"string","nullable":true,"desc":"The posting's country as **ISO 3166-1 alpha-2** — `AZ`, `PL`, `US`. **Null unless the source itself published a country**, which most do not: an Azerbaijani board's postings are Azerbaijani by construction and it never says so on the row. So null means *nobody said*, NOT *somewhere else* — use this to narrow what you serve, never to conclude a posting is foreign.\n\nIndependent of `city`, not a widening of it. `city` is an Azerbaijani-only vocabulary, so a Warsaw posting is null there and `PL` here, while a Baku posting whose source named no country is `Bakı` there and null here.\n\nIt comes from a structured country field where the source publishes one (Oracle ATS `PrimaryLocationCountry`, jobpilot's `country_code`) and from a country component of `location` where the source appends one — the same provenance rule as `city`: nothing is inferred from the posting body, and a token that is not an ISO code is stored as null rather than published."},{"name":"remote","type":"boolean","nullable":true,"desc":"Whether the posting is remote-eligible."},{"name":"employment_type","type":"string","nullable":true,"desc":"Full-time, Part-time, Contract, as the source labels it."},{"name":"job_function","type":"string","nullable":true,"desc":"The source's own function taxonomy, not normalised across sources."},{"name":"career_level","type":"string","nullable":true,"desc":"The source's own seniority label. Not normalised, so it is not called \"seniority\"."},{"name":"salary_from","type":"number","nullable":true,"desc":"Bottom of the published pay range, in `salary_currency`. Null where the source does not publish pay, which is most postings -- and null rather than 0, which would be a figure. Nothing is inferred from the posting body."},{"name":"salary_to","type":"number","nullable":true,"desc":"Top of the range. Either end can be null on its own: plenty of postings advertise a floor with no ceiling."},{"name":"salary_currency","type":"string","nullable":true,"desc":"The source's own currency code for the range, usually AZN. Not converted, so two postings in different currencies are not comparable by number alone."},{"name":"salary_period","type":"string","nullable":true,"desc":"What the figures are per: `HOUR`, `DAY`, `WEEK`, `MONTH` or `YEAR`. **Null unless the source itself stated it**, which today is about 1% of live postings and 6% of those that carry pay at all, and always `MONTH` — the eight sources that publish a period all publish that one. Null is the useful answer, not a gap: it means nobody knows, so do not assume. If you emit JobPosting `unitText`, omit it when this is null rather than defaulting to `MONTH`; a figure published under the wrong period is wrong by 12x. Nothing here is inferred from the amount, and nothing is read out of the posting body — a period word near a salary is usually about a shift, a rest day or required experience, not about pay."},{"name":"deadline_at","type":"timestamp","nullable":true,"desc":"Application deadline as published, RFC 3339. Unlike `posted_at` a future date is kept rather than nulled -- a date the clock has not reached is what a deadline is. Not a liveness signal: a posting is not withdrawn here when its deadline passes, and most postings have none."},{"name":"posted_at","type":"timestamp","nullable":true,"desc":"When the posting first appeared, RFC 3339."},{"name":"url","type":"string","nullable":true,"desc":"The posting's page on the source site — usually canonical, but see the last paragraph: on some sources it is a third-party redirect carrying search-session state rather than a canonical page. **Not unique**, in two different ways, and **not stable over time** in a third.\n\n**Same source.** A board that lists one vacancy under two of its own ids yields two rows that differ by `id` and match by `url`, and they can land in the SAME page of a collection — so a consumer keying a table on `url` must deduplicate before writing, or one duplicated pair will reject the whole statement and take the other 98 rows of the page with it. Collapse these by keeping the newer `ingested_at`.\n\n**Different sources.** An aggregator republishes another board's posting under its own id, carrying the ORIGINAL board's url. You then hold two records for one job with the same `url`, different `source`, and different `id`. **This pair cannot be collapsed the same way** — the two disagree about which `\u003csource\u003e:\u003csource_job_id\u003e` addresses the posting, so whichever you keep decides the id you can look it up by afterwards. Pick the record whose `source` owns the url's domain, or skip aggregators entirely: `GET /v1/sources` marks them with `\"aggregator\": true`.\n\n**Over time.** The same `id` can return a DIFFERENT `url` on any scrape, so a mirror cannot key on it — this is the arity the other two paragraphs do not cover, and unlike them it cannot be absorbed by deduplicating a page. Measured on `jobpilot_remote`: one id emitted **39 distinct urls over five weeks** for one unchanged posting. Both the path and the query rotate (`/away/` vs `/desc/`), and `jobAge` increments by exactly 24 every scrape, so the url is **guaranteed** to differ on every daily run. It is source-specific rather than fleet-wide: 11,722 of 11,722 sampled active urls on that source carry tracking parameters, against 0 of 100 on each of boss.az, tap.az, jobsearch.az and Glorri. Those redirect urls are also not canonical pages — `ckey=` is the search query somebody typed, not part of the posting's address.\n\nSo: **`id` is the identifier, and this is a link that moves.** Key on `id`, store `url` as an attribute, and if you must group by url, strip the query string first."},{"name":"description","type":"string","nullable":true,"desc":"Full posting body. Present when the crawler has fetched the posting's own page, null otherwise — on collections and on /v1/jobs/{id} alike, since neither fetches on demand.\n\n**92% of live postings**, counted over a complete walk of them on 2026-08-28, and 97% on `source_type=employer,platform,agency`. The gap is per-source and mostly upstream: a handful of sites give a listing and no reachable detail page. It is markedly worse on RETIRED postings, so a walk left at today's default `?active=any` will look thinner than the live data actually is."},{"name":"requirements","type":"string","nullable":true,"desc":"Requirements section when the source separates it, subject to the same condition as description. **Present on about 11% of live postings** (this said 5%, measured against a different denominator) — most boards write one prose block and it lands in `description`. Concentrated rather than spread: Glorri separates it on virtually every posting and boss.az on most, and the rest of the fleet barely at all. Do not build a layout that needs this."},{"name":"description_html","type":"string","nullable":true,"desc":"Markup for the body — `\u003cp\u003e`, `\u003cul\u003e`, `\u003cli\u003e`. **Only returned by `GET /v1/jobs/{id}`.** It is absent, not null, on collections: a body roughly doubles the size of a row that has one, so a walk of 100 does not carry it. Fetch the ids you need it for.\n\n**Two provenances, and you cannot tell them apart from the field alone.** Where the source published HTML — about 30 of our 130 sources — this is that source's own markup, passed through, and the only changes we make to it are cosmetic: script and style elements removed at capture, a block element holding nothing but a space or an `\u0026nbsp;` dropped, and a bullet glyph glued to its first word given the space back. Where the source published only plain text, **this markup is ours**, generated from `description` — paragraphs become `\u003cp\u003e`, a run of bullet lines becomes a real `\u003cul\u003e`/`\u003cli\u003e`, and everything is escaped. It carries no information `description` does not; what it gives you is one shape to render whichever scraper produced the row.\n\nGenerated markup never replaces a source's own: it only ever fills a value that was empty. A row not re-observed since 2026-08-31 can still be null — there is no backfill, for the reason recorded in migration 0037. **Escape it before rendering either way**: the pass-through half is a stranger's HTML."},{"name":"requirements_html","type":"string","nullable":true,"desc":"As `description_html`, for the requirements section where the source separates it. Same two provenances, same condition, same endpoint."},{"name":"facts","type":"object","nullable":true,"desc":"The upstream columns this platform does not publish as their own field, under one key because only a handful of sources fill them. Five of the keys the live fleet fills — `work_type`, `company_logo`, `view_count`, `contact_email` and `contact_phone` — are typed fields of their own now and are still echoed here as strings; the typed fields are the ones to read, and `view_count` in particular can appear here as an abbreviated string like `1.3K` that will not cast. `slug` is the source's own. The two contact keys are rare — see `contact_email` for where each one comes from and for the two conditions an address named in a posting body has to meet before it is published. Note that a minority of rows imported from the upstream board's database carry an address THAT board extracted from advert text before the import, under rules this platform does not apply to its own crawling. They are the employer's own application contact for that posting, given for applications: do not use them to build a contact list, and do not send anything to them that is not a reply to the posting they came with. Postings imported from the upstream job board's own database — a large part of the archive — carry more, including that board's internal bookkeeping (`is_active`, `flag_status`, `created_at` and `click_count`). **Do not read those back.** They are the origin board's own columns about its own copy, frozen at the moment we imported its database — a mirror reading `facts.is_active` would resurrect its own retired rows from its own stale data. The `active` field is this platform's answer and the only one that moves. Treat the key set as per-row and per-source rather than as a schema: check for a key, do not assume it. Every value is rendered as a string whatever its underlying type, and a nested object or array is dropped rather than serialised. Unlike every other nullable field here the key is ABSENT rather than null on a posting with none of them. Salary and the deadline used to live here and are now typed fields of their own; they are still echoed inside `facts` as strings, and the typed columns are the ones to read."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored. Moves every time a scrape re-sees the posting."},{"name":"first_seen_at","type":"timestamp","nullable":false,"desc":"When this platform first saw the posting. Never changes — this is what distinguishes a new posting from one you already hold, since ingested_at moves on every re-sighting."},{"name":"times_seen","type":"integer","nullable":false,"desc":"How many scrapes have seen this posting. A source whose rows are all at 1 is changing its own identifiers rather than posting new work."},{"name":"active","type":"boolean","nullable":false,"desc":"**A verdict we can only give when the source has run.** `false` is a real finding: the source delivered a complete scrape, this posting was not in it, and nothing had seen it for 24 hours — roughly three consecutive misses on the published schedule.\n\n`true` is **weaker than it looks, and you should not read it as \"verified live today\"**. It is the default state, and the only thing that ever changes it is the retirement pass — which fires only when the source itself completes a run. A source that has stopped being scraped produces no verdict at all, so every one of its postings stays `true` indefinitely, and this description used to illustrate that with a number that was simply wrong — `jobsite_az` holding \"over 12,000 postings, all still `true`\". It holds 12,667 rows of which **1,199 are active**; the other 11,468 were retired on 2026-08-12, before that sentence was ever written. The figure was a corpus total relabelled as an active count.\n\n**You no longer have to take our word for the size of it.** `GET /v1/sources` publishes `live_records` beside `records` per source, so the gap between the two is exactly the population whose `true` carries no verdict — and a source whose `last_ingested_at` is old is one where that gap is unmeasured rather than small. Two sources report thousands of `records` and zero `live_records`.\n\n**If liveness matters to you, derive it yourself** — you have everything you need and it is four lines. Compare this posting's `ingested_at` against its source's `last_ingested_at` from `GET /v1/sources`: the source ran recently AND carried this row means live; the source ran recently and did NOT means gone; the source has not run means nobody knows, including us. See the worked example in the freshness docs. Nothing stops you being stricter than we are, and for a jobseeker-facing product you probably should be."},{"name":"deactivated_at","type":"timestamp","nullable":true,"desc":"When this platform concluded the posting had gone, RFC 3339, and null while it is active. Not when the employer took it down — nothing here can observe that — but at most one grace window later. `ingested_at` deliberately does not move when a posting is retired, so this is the timestamp that changed; `updated_after` matches on either."},{"name":"work_type","type":"string","nullable":true,"desc":"The source's own working-arrangement label — on-site, hybrid, shift — as it writes it. Not normalised, and not the same axis as `employment_type`, which is about the contract. Null on most postings."},{"name":"company_logo","type":"string","nullable":true,"desc":"Absolute URL of the company's logo on the source's own site. Not mirrored and not revalidated, so it can 404 independently of the posting — treat a broken image as expected rather than as an error."},{"name":"view_count","type":"integer","nullable":true,"desc":"How many times the SOURCE says the posting has been viewed — its counter, not ours, and not comparable between sources. Approximate where the source abbreviated it: a board showing 1.3K has published somewhere in 1,250–1,349, and that precision is gone before it reaches us. Null where the source publishes no counter."},{"name":"contact_email","type":"string","nullable":true,"desc":"**About 6% of live postings, and the distribution matters more than the number.** About 85% of these rows are on job boards, nearly all of that on four of them — jobsearch.az, boss.az, work.az and position.az — because a board's posting form asks the employer for a contact address. On `source_type=employer` it is about 28%, and on a board like boss.az it is every row. **If you filter to direct employers and also need an address, those two requirements pull against each other**: a company's own careers page routes applications through its own form or ATS, so most such postings publish no address at all and `url` is the apply path.\n\nWhere it is present it is the employer's own application mailbox for this posting, lower-cased, from one of two places and no others:\n\n1. a **structured field** the source published — an API field, a `mailto:` or a `tel:` link — read by nine of the ~120 scrapers;\n2. an **application mailbox named in the posting itself**, under three conditions that all have to hold: the **whole local part** is a role name from a closed list (`cv@`, `hr@`, `karyera@`, `vacancy@`, `tecrube@` …) with nothing appended, so `hr.aliyeva@` does not qualify; the **domain is the posting's own**; and the posting comes from a source whose url identifies **one organisation** — a direct employer or a named recruitment agency, never a job board, a multi-tenant careers platform or a hosted ATS domain shared between employers. Together those mean a named individual's address cannot qualify and neither can a third party's. Ten distinct mailboxes account for every row filled this way.\n\nNothing else is read out of advert prose, deliberately. Use it to reply to the posting it arrived with; do not use it to build a contact list."},{"name":"contact_phone","type":"string","nullable":true,"desc":"**About 4% of live postings**, and 98% of those are on jobsearch.az and boss.az alone — on employer career pages it is effectively absent (one row in the whole segment). Formatted exactly as the source published it, with no canonicalisation, because a number's formatting is theirs and normalising it here would be inventing one. Do not assume E.164.\n\nUnlike `contact_email` this comes ONLY from a structured field — an API field or a `tel:` link. Nothing is read from the posting body, and the asymmetry is deliberate: a number has no local part, so there is no way to tell an employer's application line from someone's mobile the way a role mailbox can be told from a person's. Across a complete walk of the live corpus there were no `tel:` links in any posting body, so there is nothing being withheld."}]},{"id":"job-liveness","verb":"POST","path":"/v1/jobs/liveness","summary":"Batch liveness","returns":"array","params":[],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Echoed back exactly as you sent it, including an id this platform could not parse. Match on it; it is not normalised."},{"name":"state","type":"string","nullable":false,"desc":"One of `active`, `inactive`, `missing`, `unknown`, and that set is closed.\n\n`active` — the posting is here and its source still lists it. `inactive` — it is here and its source's last complete run did not list it. `missing` — this platform holds no such posting, either because it never did or because it was withdrawn on request; both are \"gone\" to a caller reconciling their own copy. `unknown` — no verdict, because the id could not be resolved into one. Treat `unknown` as \"change nothing, ask again\"; it is the safe answer and it is why every id gets a row rather than being dropped."},{"name":"active","type":"boolean","nullable":true,"desc":"A real boolean whenever `state` is `active` or `inactive`, and null otherwise. Never absent."},{"name":"checked_at","type":"string (RFC 3339)","nullable":false,"desc":"When this verdict was read. One timestamp for the whole batch, because it is one query."}]},{"id":"list-company-jobs","verb":"GET","path":"/v1/companies/{id}/jobs","summary":"Company jobs","returns":"array","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"Company slug or name.","example":"fincaazerbaijan"},{"name":"q","in":"query","type":"string","required":false,"desc":"Full-text match within this company's postings."},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, `posted_at`, each optionally prefixed with `-` to reverse. Default is `-posted_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-posted_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\". Pass it to /v1/jobs/{id}."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper produced this posting. A stable slug, and half of `id` — this is what `?source=` takes, so it never changes even when a board renames itself. `GET /v1/sources` lists every slug with its display name, its dataset, how many rows it currently has and when it last landed a batch. That endpoint needs no key."},{"name":"source_name","type":"string","nullable":true,"desc":"The same source as a person would write it — `ABB` for `abb`, `1is.az` for `1is_az`. This is the one to display. Null for the handful of sources the scraper registry has no entry for, rather than repeating the slug."},{"name":"source_url","type":"string","nullable":true,"desc":"The source's own website — `https://glorri.com` for `glorri`, `https://abb-bank.az` for `abb`. This is the one to link `source_name` to.\n\n**Not this posting's link** — that is `url` — and not the source's listing page either. `GET /v1/sources` carries both, as `url` and `jobs_url`, and needs no key; `jobs_url` there is the page this platform reads the source's vacancies from, which is the better link if you are sending somebody to browse a source rather than to read about it.\n\nSet for 122 of 130 sources. Null on eight rather than guessed: three are hosted entirely on a third party's applicant-tracking system and their own domain appears nowhere in our scrapers (their `jobs_url` on `/v1/sources` still points at the ATS page the postings are actually on), one has only its ATS vendor's domain recorded against it, `other` is a catch-all bucket with no website, and **three we have measured as gone** — a hosting placeholder on every path, an organisation that no longer exists and now redirects to a different agency, and a decommissioned ATS tenant. Those three keep their rows and lose their link, because a link to nothing is worse than none."},{"name":"title","type":"string","nullable":false,"desc":"Posting title as published."},{"name":"company","type":"object","nullable":true,"desc":"The hiring company. `company.name` is always present when the object is; `company.slug` is nullable and is null on most rows -- do not type it as required. The whole object is null when the source omits the company, and is ALWAYS null on classifieds sources: there the advert is placed by a private individual rather than an employer, so there is no hiring company to name and publishing the poster under this field would be publishing a person's name as a company (#83)."},{"name":"location","type":"string","nullable":true,"desc":"Free text at whatever precision the source itself publishes — a city, a city and country, a district, an office or branch name, or the country alone. Not normalised, not geocoded, and not comparable between sources. Some boards publish no city at all and this is their country; nothing is inferred from the posting body to fill the gap, because a city read out of prose is as likely to be the employer's head office as the job's location."},{"name":"city","type":"string","nullable":true,"desc":"**25% of live postings** — 9,690 of 39,046, counted exactly on 2026-08-28 by summing `?city=` across the whole vocabulary. This said 60% and climbing, which was a projection rather than a measurement and is not where it is heading: the field fills only from sources that publish a structured place, about twenty of a hundred and thirty do, and several of the largest sources never will. Nine in ten of the values are `Bakı`. The same place as `location`, normalised to a closed vocabulary of Azerbaijani cities so it can be filtered and grouped — which `location` cannot be. **Null unless the source published a place of its own**: a city inferred from advert prose is as likely to be the employer's head office as the job's, so postings whose location was recovered that way carry a null city and keep their `location`. Azerbaijani cities only — a posting in Moscow or Warsaw is null here and unchanged in `location`. `location` is untouched and stays the higher-precision value; this is the comparable one."},{"name":"district","type":"string","nullable":true,"desc":"The Baku rayon, where the source names one — `Nəsimi`, `Binəqədi`, `Xətai` and the other nine. Null everywhere else, including for Baku postings that name no rayon. Implies `city`: a rayon is Baku's, so `city` is always `Bakı` when this is set. Present because one source publishes a rayon where every other publishes a city, and because a per-rayon page needs rayon granularity that a city vocabulary cannot express."},{"name":"country","type":"string","nullable":true,"desc":"The posting's country as **ISO 3166-1 alpha-2** — `AZ`, `PL`, `US`. **Null unless the source itself published a country**, which most do not: an Azerbaijani board's postings are Azerbaijani by construction and it never says so on the row. So null means *nobody said*, NOT *somewhere else* — use this to narrow what you serve, never to conclude a posting is foreign.\n\nIndependent of `city`, not a widening of it. `city` is an Azerbaijani-only vocabulary, so a Warsaw posting is null there and `PL` here, while a Baku posting whose source named no country is `Bakı` there and null here.\n\nIt comes from a structured country field where the source publishes one (Oracle ATS `PrimaryLocationCountry`, jobpilot's `country_code`) and from a country component of `location` where the source appends one — the same provenance rule as `city`: nothing is inferred from the posting body, and a token that is not an ISO code is stored as null rather than published."},{"name":"remote","type":"boolean","nullable":true,"desc":"Whether the posting is remote-eligible."},{"name":"employment_type","type":"string","nullable":true,"desc":"Full-time, Part-time, Contract, as the source labels it."},{"name":"job_function","type":"string","nullable":true,"desc":"The source's own function taxonomy, not normalised across sources."},{"name":"career_level","type":"string","nullable":true,"desc":"The source's own seniority label. Not normalised, so it is not called \"seniority\"."},{"name":"salary_from","type":"number","nullable":true,"desc":"Bottom of the published pay range, in `salary_currency`. Null where the source does not publish pay, which is most postings -- and null rather than 0, which would be a figure. Nothing is inferred from the posting body."},{"name":"salary_to","type":"number","nullable":true,"desc":"Top of the range. Either end can be null on its own: plenty of postings advertise a floor with no ceiling."},{"name":"salary_currency","type":"string","nullable":true,"desc":"The source's own currency code for the range, usually AZN. Not converted, so two postings in different currencies are not comparable by number alone."},{"name":"salary_period","type":"string","nullable":true,"desc":"What the figures are per: `HOUR`, `DAY`, `WEEK`, `MONTH` or `YEAR`. **Null unless the source itself stated it**, which today is about 1% of live postings and 6% of those that carry pay at all, and always `MONTH` — the eight sources that publish a period all publish that one. Null is the useful answer, not a gap: it means nobody knows, so do not assume. If you emit JobPosting `unitText`, omit it when this is null rather than defaulting to `MONTH`; a figure published under the wrong period is wrong by 12x. Nothing here is inferred from the amount, and nothing is read out of the posting body — a period word near a salary is usually about a shift, a rest day or required experience, not about pay."},{"name":"deadline_at","type":"timestamp","nullable":true,"desc":"Application deadline as published, RFC 3339. Unlike `posted_at` a future date is kept rather than nulled -- a date the clock has not reached is what a deadline is. Not a liveness signal: a posting is not withdrawn here when its deadline passes, and most postings have none."},{"name":"posted_at","type":"timestamp","nullable":true,"desc":"When the posting first appeared, RFC 3339."},{"name":"url","type":"string","nullable":true,"desc":"The posting's page on the source site — usually canonical, but see the last paragraph: on some sources it is a third-party redirect carrying search-session state rather than a canonical page. **Not unique**, in two different ways, and **not stable over time** in a third.\n\n**Same source.** A board that lists one vacancy under two of its own ids yields two rows that differ by `id` and match by `url`, and they can land in the SAME page of a collection — so a consumer keying a table on `url` must deduplicate before writing, or one duplicated pair will reject the whole statement and take the other 98 rows of the page with it. Collapse these by keeping the newer `ingested_at`.\n\n**Different sources.** An aggregator republishes another board's posting under its own id, carrying the ORIGINAL board's url. You then hold two records for one job with the same `url`, different `source`, and different `id`. **This pair cannot be collapsed the same way** — the two disagree about which `\u003csource\u003e:\u003csource_job_id\u003e` addresses the posting, so whichever you keep decides the id you can look it up by afterwards. Pick the record whose `source` owns the url's domain, or skip aggregators entirely: `GET /v1/sources` marks them with `\"aggregator\": true`.\n\n**Over time.** The same `id` can return a DIFFERENT `url` on any scrape, so a mirror cannot key on it — this is the arity the other two paragraphs do not cover, and unlike them it cannot be absorbed by deduplicating a page. Measured on `jobpilot_remote`: one id emitted **39 distinct urls over five weeks** for one unchanged posting. Both the path and the query rotate (`/away/` vs `/desc/`), and `jobAge` increments by exactly 24 every scrape, so the url is **guaranteed** to differ on every daily run. It is source-specific rather than fleet-wide: 11,722 of 11,722 sampled active urls on that source carry tracking parameters, against 0 of 100 on each of boss.az, tap.az, jobsearch.az and Glorri. Those redirect urls are also not canonical pages — `ckey=` is the search query somebody typed, not part of the posting's address.\n\nSo: **`id` is the identifier, and this is a link that moves.** Key on `id`, store `url` as an attribute, and if you must group by url, strip the query string first."},{"name":"description","type":"string","nullable":true,"desc":"Full posting body. Present when the crawler has fetched the posting's own page, null otherwise — on collections and on /v1/jobs/{id} alike, since neither fetches on demand.\n\n**92% of live postings**, counted over a complete walk of them on 2026-08-28, and 97% on `source_type=employer,platform,agency`. The gap is per-source and mostly upstream: a handful of sites give a listing and no reachable detail page. It is markedly worse on RETIRED postings, so a walk left at today's default `?active=any` will look thinner than the live data actually is."},{"name":"requirements","type":"string","nullable":true,"desc":"Requirements section when the source separates it, subject to the same condition as description. **Present on about 11% of live postings** (this said 5%, measured against a different denominator) — most boards write one prose block and it lands in `description`. Concentrated rather than spread: Glorri separates it on virtually every posting and boss.az on most, and the rest of the fleet barely at all. Do not build a layout that needs this."},{"name":"facts","type":"object","nullable":true,"desc":"The upstream columns this platform does not publish as their own field, under one key because only a handful of sources fill them. Five of the keys the live fleet fills — `work_type`, `company_logo`, `view_count`, `contact_email` and `contact_phone` — are typed fields of their own now and are still echoed here as strings; the typed fields are the ones to read, and `view_count` in particular can appear here as an abbreviated string like `1.3K` that will not cast. `slug` is the source's own. The two contact keys are rare — see `contact_email` for where each one comes from and for the two conditions an address named in a posting body has to meet before it is published. Note that a minority of rows imported from the upstream board's database carry an address THAT board extracted from advert text before the import, under rules this platform does not apply to its own crawling. They are the employer's own application contact for that posting, given for applications: do not use them to build a contact list, and do not send anything to them that is not a reply to the posting they came with. Postings imported from the upstream job board's own database — a large part of the archive — carry more, including that board's internal bookkeeping (`is_active`, `flag_status`, `created_at` and `click_count`). **Do not read those back.** They are the origin board's own columns about its own copy, frozen at the moment we imported its database — a mirror reading `facts.is_active` would resurrect its own retired rows from its own stale data. The `active` field is this platform's answer and the only one that moves. Treat the key set as per-row and per-source rather than as a schema: check for a key, do not assume it. Every value is rendered as a string whatever its underlying type, and a nested object or array is dropped rather than serialised. Unlike every other nullable field here the key is ABSENT rather than null on a posting with none of them. Salary and the deadline used to live here and are now typed fields of their own; they are still echoed inside `facts` as strings, and the typed columns are the ones to read."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored. Moves every time a scrape re-sees the posting."},{"name":"first_seen_at","type":"timestamp","nullable":false,"desc":"When this platform first saw the posting. Never changes — this is what distinguishes a new posting from one you already hold, since ingested_at moves on every re-sighting."},{"name":"times_seen","type":"integer","nullable":false,"desc":"How many scrapes have seen this posting. A source whose rows are all at 1 is changing its own identifiers rather than posting new work."},{"name":"active","type":"boolean","nullable":false,"desc":"**A verdict we can only give when the source has run.** `false` is a real finding: the source delivered a complete scrape, this posting was not in it, and nothing had seen it for 24 hours — roughly three consecutive misses on the published schedule.\n\n`true` is **weaker than it looks, and you should not read it as \"verified live today\"**. It is the default state, and the only thing that ever changes it is the retirement pass — which fires only when the source itself completes a run. A source that has stopped being scraped produces no verdict at all, so every one of its postings stays `true` indefinitely, and this description used to illustrate that with a number that was simply wrong — `jobsite_az` holding \"over 12,000 postings, all still `true`\". It holds 12,667 rows of which **1,199 are active**; the other 11,468 were retired on 2026-08-12, before that sentence was ever written. The figure was a corpus total relabelled as an active count.\n\n**You no longer have to take our word for the size of it.** `GET /v1/sources` publishes `live_records` beside `records` per source, so the gap between the two is exactly the population whose `true` carries no verdict — and a source whose `last_ingested_at` is old is one where that gap is unmeasured rather than small. Two sources report thousands of `records` and zero `live_records`.\n\n**If liveness matters to you, derive it yourself** — you have everything you need and it is four lines. Compare this posting's `ingested_at` against its source's `last_ingested_at` from `GET /v1/sources`: the source ran recently AND carried this row means live; the source ran recently and did NOT means gone; the source has not run means nobody knows, including us. See the worked example in the freshness docs. Nothing stops you being stricter than we are, and for a jobseeker-facing product you probably should be."},{"name":"deactivated_at","type":"timestamp","nullable":true,"desc":"When this platform concluded the posting had gone, RFC 3339, and null while it is active. Not when the employer took it down — nothing here can observe that — but at most one grace window later. `ingested_at` deliberately does not move when a posting is retired, so this is the timestamp that changed; `updated_after` matches on either."},{"name":"work_type","type":"string","nullable":true,"desc":"The source's own working-arrangement label — on-site, hybrid, shift — as it writes it. Not normalised, and not the same axis as `employment_type`, which is about the contract. Null on most postings."},{"name":"company_logo","type":"string","nullable":true,"desc":"Absolute URL of the company's logo on the source's own site. Not mirrored and not revalidated, so it can 404 independently of the posting — treat a broken image as expected rather than as an error."},{"name":"view_count","type":"integer","nullable":true,"desc":"How many times the SOURCE says the posting has been viewed — its counter, not ours, and not comparable between sources. Approximate where the source abbreviated it: a board showing 1.3K has published somewhere in 1,250–1,349, and that precision is gone before it reaches us. Null where the source publishes no counter."},{"name":"contact_email","type":"string","nullable":true,"desc":"**About 6% of live postings, and the distribution matters more than the number.** About 85% of these rows are on job boards, nearly all of that on four of them — jobsearch.az, boss.az, work.az and position.az — because a board's posting form asks the employer for a contact address. On `source_type=employer` it is about 28%, and on a board like boss.az it is every row. **If you filter to direct employers and also need an address, those two requirements pull against each other**: a company's own careers page routes applications through its own form or ATS, so most such postings publish no address at all and `url` is the apply path.\n\nWhere it is present it is the employer's own application mailbox for this posting, lower-cased, from one of two places and no others:\n\n1. a **structured field** the source published — an API field, a `mailto:` or a `tel:` link — read by nine of the ~120 scrapers;\n2. an **application mailbox named in the posting itself**, under three conditions that all have to hold: the **whole local part** is a role name from a closed list (`cv@`, `hr@`, `karyera@`, `vacancy@`, `tecrube@` …) with nothing appended, so `hr.aliyeva@` does not qualify; the **domain is the posting's own**; and the posting comes from a source whose url identifies **one organisation** — a direct employer or a named recruitment agency, never a job board, a multi-tenant careers platform or a hosted ATS domain shared between employers. Together those mean a named individual's address cannot qualify and neither can a third party's. Ten distinct mailboxes account for every row filled this way.\n\nNothing else is read out of advert prose, deliberately. Use it to reply to the posting it arrived with; do not use it to build a contact list."},{"name":"contact_phone","type":"string","nullable":true,"desc":"**About 4% of live postings**, and 98% of those are on jobsearch.az and boss.az alone — on employer career pages it is effectively absent (one row in the whole segment). Formatted exactly as the source published it, with no canonicalisation, because a number's formatting is theirs and normalising it here would be inventing one. Do not assume E.164.\n\nUnlike `contact_email` this comes ONLY from a structured field — an API field or a `tel:` link. Nothing is read from the posting body, and the asymmetry is deliberate: a number has no local part, so there is no way to tell an employer's application line from someone's mobile the way a role mailbox can be told from a person's. Across a complete walk of the live corpus there were no `tel:` links in any posting body, so there is nothing being withheld."}]}]},{"slug":"events","name":"Event Data API","status":"live","operations":[{"id":"list-events","verb":"GET","path":"/v1/events","summary":"List events","returns":"array","params":[{"name":"q","in":"query","type":"string","required":false,"desc":"Full-text match on name, venue and description.","example":"gimnastika"},{"name":"category","in":"query","type":"string","required":false,"desc":"Exact match on the source's own category slug, which is not a closed set — see the field description. `GET /v1/events` with no filter is the only reliable way to learn which values exist today.","example":"cinema"},{"name":"updated_after","in":"query","type":"timestamp","required":false,"desc":"Only rows whose `ingested_at` is later than this instant — everything that CHANGED since your last poll, new and re-seen alike. Pass the newest `ingested_at` you have stored. This is what a mirror of the collection should page on; `starts_after` filters on when the EVENT happens, which is a different question.","example":"2026-08-16T09:00:00Z"},{"name":"starts_after","in":"query","type":"timestamp","required":false,"desc":"Events starting at or after this instant."},{"name":"starts_before","in":"query","type":"timestamp","required":false,"desc":"Events starting at or before this instant."},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, `starts_at`, each optionally prefixed with `-` to reverse. Default is `starts_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"starts_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\"."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper produced this event."},{"name":"slug","type":"string","nullable":true,"desc":"The source's own readable identifier for the event — what to build a stable, human-readable URL from, rather than the opaque `id`. NOT unique and not an identifier: one source derives it from a film title and a cinema name, so the same film at two cinemas shares a slug. Null where the source publishes none, and a value that stops arriving is kept rather than nulled, because a stale slug is a working link and an empty one is a 404."},{"name":"name","type":"string","nullable":false,"desc":"Event name as published."},{"name":"category","type":"string","nullable":true,"desc":"The source's own category slug, not normalised and not a closed set — it follows the upstream, so it also contains venue and festival names (`hayal-kahvesi`, `gabalafest`). `cinema`, `concerts` and `theatre` are three quarters of the corpus between them; `kids`, `sport`, `seminar`, `master-class`, `circus`, `other` and `exhibitions` make up most of the rest. Do not build a fixed facet from this list — read the values you actually receive."},{"name":"venue","type":"object","nullable":true,"desc":"id, name, address, lat, lng. Coordinates where the source geocodes."},{"name":"starts_at","type":"timestamp","nullable":true,"desc":"Start time as an RFC 3339 instant in UTC. Convert with the venue's timezone for display — all current venues are Asia/Baku (UTC+4)."},{"name":"ends_at","type":"timestamp","nullable":true,"desc":"End time in UTC, when the source publishes one."},{"name":"price","type":"object","nullable":true,"desc":"min, max and currency across ticket tiers."},{"name":"age_limit","type":"string","nullable":true,"desc":"Age restriction as published, e.g. \"16+\"."},{"name":"tickets_available","type":"integer","nullable":true,"desc":"Live inventory. null where the source does not expose it — never 0 as a stand-in."},{"name":"description","type":"string","nullable":true,"desc":"Editorial copy for the event."},{"name":"url","type":"string","nullable":true,"desc":"Canonical page on the source site."},{"name":"external_url","type":"string","nullable":true,"desc":"Information or ticketing page. NOT guaranteed to be a purchase link — cinema sources have no dereferenceable buy href."},{"name":"poster_url","type":"string","nullable":true,"desc":"Artwork, highest resolution the source offers."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]},{"id":"get-event","verb":"GET","path":"/v1/events/{id}","summary":"Retrieve an event","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The event id from a list response.","example":"iticket:12299"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\"."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper produced this event."},{"name":"slug","type":"string","nullable":true,"desc":"The source's own readable identifier for the event — what to build a stable, human-readable URL from, rather than the opaque `id`. NOT unique and not an identifier: one source derives it from a film title and a cinema name, so the same film at two cinemas shares a slug. Null where the source publishes none, and a value that stops arriving is kept rather than nulled, because a stale slug is a working link and an empty one is a 404."},{"name":"name","type":"string","nullable":false,"desc":"Event name as published."},{"name":"category","type":"string","nullable":true,"desc":"The source's own category slug, not normalised and not a closed set — it follows the upstream, so it also contains venue and festival names (`hayal-kahvesi`, `gabalafest`). `cinema`, `concerts` and `theatre` are three quarters of the corpus between them; `kids`, `sport`, `seminar`, `master-class`, `circus`, `other` and `exhibitions` make up most of the rest. Do not build a fixed facet from this list — read the values you actually receive."},{"name":"venue","type":"object","nullable":true,"desc":"id, name, address, lat, lng. Coordinates where the source geocodes."},{"name":"starts_at","type":"timestamp","nullable":true,"desc":"Start time as an RFC 3339 instant in UTC. Convert with the venue's timezone for display — all current venues are Asia/Baku (UTC+4)."},{"name":"ends_at","type":"timestamp","nullable":true,"desc":"End time in UTC, when the source publishes one."},{"name":"price","type":"object","nullable":true,"desc":"min, max and currency across ticket tiers."},{"name":"age_limit","type":"string","nullable":true,"desc":"Age restriction as published, e.g. \"16+\"."},{"name":"tickets_available","type":"integer","nullable":true,"desc":"Live inventory. null where the source does not expose it — never 0 as a stand-in."},{"name":"description","type":"string","nullable":true,"desc":"Editorial copy for the event."},{"name":"url","type":"string","nullable":true,"desc":"Canonical page on the source site."},{"name":"external_url","type":"string","nullable":true,"desc":"Information or ticketing page. NOT guaranteed to be a purchase link — cinema sources have no dereferenceable buy href."},{"name":"poster_url","type":"string","nullable":true,"desc":"Artwork, highest resolution the source offers."},{"name":"sessions","type":"array","nullable":true,"desc":"Every individual showing, each with its own price range and inventory. Returned by `GET /v1/events/{id}` only, **and only where the source publishes per-showing inventory** — which is often not the case. The key is ABSENT rather than null when it does not, so check for it before iterating: one of the two current sources publishes none at all, and the other omits it on about a third of its events. `tickets_available` is null on the same rows."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."},{"name":"sessions[].id","type":"string","nullable":false,"desc":"Session identifier from the source."},{"name":"sessions[].starts_at","type":"timestamp","nullable":true,"desc":"When this showing starts, UTC."},{"name":"sessions[].ends_at","type":"timestamp","nullable":true,"desc":"When it ends, when the source publishes one."},{"name":"sessions[].sell_ends_at","type":"timestamp","nullable":true,"desc":"When sales close for this showing."},{"name":"sessions[].price","type":"object","nullable":true,"desc":"min, max and currency for this showing, which can differ from the event's overall range."},{"name":"sessions[].tickets_available","type":"integer","nullable":true,"desc":"Remaining inventory for THIS showing at observed_at, not the event total."},{"name":"sessions[].language","type":"string","nullable":true,"desc":"Presentation language. null where the source does not label it."},{"name":"sessions[].observed_at","type":"timestamp","nullable":true,"desc":"When this inventory figure was scraped — successive observations form a sell-through time series."}]},{"id":"get-venue","verb":"GET","path":"/v1/venues/{id}","summary":"Retrieve a venue","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"Venue slug from an event's venue.id.","example":"museum-center"}],"fields":[{"name":"id","type":"string","nullable":true,"desc":"Slugified venue name. Venues have no source-issued id."},{"name":"name","type":"string","nullable":false,"desc":"Venue name as published."},{"name":"address","type":"string","nullable":true,"desc":"Street address as published."},{"name":"lat","type":"number","nullable":true,"desc":"Latitude, WGS 84."},{"name":"lng","type":"number","nullable":true,"desc":"Longitude, WGS 84."}]}]},{"slug":"quotes","name":"Quotes Data API","status":"live","operations":[{"id":"list-quotes","verb":"GET","path":"/v1/quotes","summary":"List quotations","returns":"array","params":[{"name":"q","in":"query","type":"string","required":false,"desc":"Full-text match on the quotation and its author.","example":"courage"},{"name":"author","in":"query","type":"string","required":false,"desc":"Author name."},{"name":"category","in":"query","type":"string","required":false,"desc":"The upstream's own label."},{"name":"active","in":"query","type":"string","required":false,"desc":"Which half of the pool. `true` (the default) is the curated quotations; `false` is the bulk archive imported from the upstream and never published there; `any` is both. The archive is large and unvetted -- some of it is a mis-parsed CSV whose author field holds part of the quotation -- which is why it is opt-in rather than the default.","example":"any"},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, each optionally prefixed with `-` to reverse. Default is `-ingested_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-ingested_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\"."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this quotation."},{"name":"text","type":"string","nullable":false,"desc":"The quotation."},{"name":"author","type":"string","nullable":true,"desc":"Attributed author. null where the collection does not name one."},{"name":"category","type":"string","nullable":true,"desc":"The upstream's own label. A folksonomy, not an enum -- tens of thousands of distinct values -- so treat it as a tag rather than a fixed set."},{"name":"origin","type":"string","nullable":true,"desc":"Where the quotation was collected from, e.g. \"passiton\". Distinct from source, which is the scraper that delivered it."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]},{"id":"get-quote","verb":"GET","path":"/v1/quotes/{id}","summary":"Retrieve a quotation","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The quotation id from a list response. Ids are `\u003csource\u003e:\u003csource_id\u003e` and are stable for the life of the record.\n\nQuotations carry a `birjob:` prefix because that is the name of the corpus they were imported from — the same project this platform's crawler used to run under, which is why it still answers to `BirJobBot`. It is a name, not a namespace you need to handle: treat the whole string as opaque and store it as you received it. It is not going to change, because an id you already hold changing would cost you a re-import to buy us a tidier prefix.","example":"birjob:1766"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\"."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this quotation."},{"name":"text","type":"string","nullable":false,"desc":"The quotation."},{"name":"author","type":"string","nullable":true,"desc":"Attributed author. null where the collection does not name one."},{"name":"category","type":"string","nullable":true,"desc":"The upstream's own label. A folksonomy, not an enum -- tens of thousands of distinct values -- so treat it as a tag rather than a fixed set."},{"name":"origin","type":"string","nullable":true,"desc":"Where the quotation was collected from, e.g. \"passiton\". Distinct from source, which is the scraper that delivered it."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]},{"id":"get-random-quote","verb":"GET","path":"/v1/quotes/random","summary":"Quote of the day","returns":"object","params":[{"name":"date","in":"query","type":"date","required":false,"desc":"ISO-8601 date in UTC. The same date and filters always return the same quotation, so it is safe to cache for the day. Defaults to today."},{"name":"category","in":"query","type":"string","required":false,"desc":"Restrict the pool to one category."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003csource id\u003e\"."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this quotation."},{"name":"text","type":"string","nullable":false,"desc":"The quotation."},{"name":"author","type":"string","nullable":true,"desc":"Attributed author. null where the collection does not name one."},{"name":"category","type":"string","nullable":true,"desc":"The upstream's own label. A folksonomy, not an enum -- tens of thousands of distinct values -- so treat it as a tag rather than a fixed set."},{"name":"origin","type":"string","nullable":true,"desc":"Where the quotation was collected from, e.g. \"passiton\". Distinct from source, which is the scraper that delivered it."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]}]},{"slug":"rates","name":"Rates Data API","status":"live","operations":[{"id":"list-rates","verb":"GET","path":"/v1/rates","summary":"List exchange rates","returns":"array","params":[{"name":"from","in":"query","type":"date","required":false,"desc":"Earliest `date`, inclusive.","example":"2026-09-01"},{"name":"to","in":"query","type":"date","required":false,"desc":"Latest `date`, inclusive.","example":"2026-09-05"},{"name":"codes","in":"query","type":"string","required":false,"desc":"Currency or metal codes, comma-separated — `USD,EUR`. Case is normalised, and a repeated `?codes=` is unioned with the list rather than overwriting it. Omit for all 42.","example":"USD,EUR"},{"name":"kind","in":"query","type":"string","required":false,"desc":"`currency` or `metal`. The metals are the four the bank publishes per troy ounce.","example":"currency"},{"name":"carried_forward","in":"query","type":"boolean","required":false,"desc":"`false` returns only the dates the bank actually published on — a strict business-day series. `true` returns only the repeats. Omit for the dense daily series, which is both.","example":"false"},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, each optionally prefixed with `-` to reverse. Default is `-ingested_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-ingested_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003cdate\u003e:\u003ccode\u003e\" — `cbar:2026-09-05:USD`. Opaque: treat the whole string as the id rather than parsing the halves out of it."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this bulletin. `cbar` is the Central Bank of Azerbaijan."},{"name":"date","type":"date","nullable":false,"desc":"The calendar date this row answers for, **Asia/Baku**. Azerbaijan is UTC+4 year-round with no DST, so a consumer computing \"today\" in UTC gets yesterday's date for the four hours after 20:00 Baku — which is a real bug this feed's predecessor shipped, not a hypothetical one."},{"name":"effective_date","type":"date","nullable":false,"desc":"The date the bank says these rates take effect from — the bulletin's own `ValCurs Date=`, never inferred from the date requested. On a business day it equals `date`; on a weekend, a public holiday or a future date it is the last business day before it."},{"name":"carried_forward","type":"boolean","nullable":false,"desc":"`date != effective_date`: these numbers were published earlier and are still standing. **Filter `carried_forward=false` for a strict business-day series**; take every row for a dense daily one and know which values are repeats. The upstream answers `200` with a full bulletin for every date string including weekends and dates in the future, so this flag is the only thing that distinguishes a fresh publication from a repeat."},{"name":"kind","type":"string","nullable":false,"desc":"`currency` or `metal`. The four bank metals (XAU, XPD, XPT, XAG) are the metals."},{"name":"base","type":"string","nullable":false,"desc":"The currency the rate is expressed in. Always `AZN` for this source."},{"name":"code","type":"string","nullable":false,"desc":"ISO 4217 code, or the metal's X-code. 42 per bulletin: 38 currencies and 4 bank metals. `SDR` is the IMF basket rather than a currency, and is carried as the bank publishes it."},{"name":"name","type":"string","nullable":true,"desc":"The bank's own label, in Azerbaijani, including the multiplier it prints there — \"1 ABŞ dolları\", \"100 Yapon iyeni\"."},{"name":"unit","type":"string","nullable":false,"desc":"What one of `nominal` is: `unit` for a currency, `troy_ounce` for a metal."},{"name":"nominal","type":"integer","nullable":false,"desc":"How many units `value` is quoted for. Seven currencies quote per 100 — KRW, KZT, HUF, UZS, PKR, RUB and JPY — and everything else per 1. Reading `value` without dividing by this is how a rouble comes out a hundred times too large."},{"name":"value","type":"number","nullable":false,"desc":"The published figure, unrounded: AZN per `nominal` units."},{"name":"rate_per_unit","type":"number","nullable":false,"desc":"`value / nominal`, unrounded. **This is the number you almost certainly want**: AZN for one unit of `code`, or for one troy ounce of a metal."},{"name":"source_url","type":"string","nullable":true,"desc":"The document requested for `date`."},{"name":"bulletin_url","type":"string","nullable":true,"desc":"The document these numbers were published in, i.e. the one for `effective_date`. Differs from `source_url` on a carried-forward row."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]},{"id":"get-latest-rates","verb":"GET","path":"/v1/rates/latest","summary":"Latest bulletin","returns":"array","params":[{"name":"from","in":"query","type":"date","required":false,"desc":"Ignore dates before this one when resolving \"latest\"."},{"name":"to","in":"query","type":"date","required":false,"desc":"Resolve \"latest\" as of this date rather than now — the bulletin that was in force on a given day."},{"name":"codes","in":"query","type":"string","required":false,"desc":"Currency or metal codes, comma-separated — `USD,EUR`.","example":"USD,EUR"},{"name":"kind","in":"query","type":"string","required":false,"desc":"`currency` or `metal`."},{"name":"carried_forward","in":"query","type":"boolean","required":false,"desc":"`false` resolves to the last date the bank actually published on, skipping a weekend's repeats."},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, each optionally prefixed with `-` to reverse. Default is `-ingested_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-ingested_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003cdate\u003e:\u003ccode\u003e\" — `cbar:2026-09-05:USD`. Opaque: treat the whole string as the id rather than parsing the halves out of it."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this bulletin. `cbar` is the Central Bank of Azerbaijan."},{"name":"date","type":"date","nullable":false,"desc":"The calendar date this row answers for, **Asia/Baku**. Azerbaijan is UTC+4 year-round with no DST, so a consumer computing \"today\" in UTC gets yesterday's date for the four hours after 20:00 Baku — which is a real bug this feed's predecessor shipped, not a hypothetical one."},{"name":"effective_date","type":"date","nullable":false,"desc":"The date the bank says these rates take effect from — the bulletin's own `ValCurs Date=`, never inferred from the date requested. On a business day it equals `date`; on a weekend, a public holiday or a future date it is the last business day before it."},{"name":"carried_forward","type":"boolean","nullable":false,"desc":"`date != effective_date`: these numbers were published earlier and are still standing. **Filter `carried_forward=false` for a strict business-day series**; take every row for a dense daily one and know which values are repeats. The upstream answers `200` with a full bulletin for every date string including weekends and dates in the future, so this flag is the only thing that distinguishes a fresh publication from a repeat."},{"name":"kind","type":"string","nullable":false,"desc":"`currency` or `metal`. The four bank metals (XAU, XPD, XPT, XAG) are the metals."},{"name":"base","type":"string","nullable":false,"desc":"The currency the rate is expressed in. Always `AZN` for this source."},{"name":"code","type":"string","nullable":false,"desc":"ISO 4217 code, or the metal's X-code. 42 per bulletin: 38 currencies and 4 bank metals. `SDR` is the IMF basket rather than a currency, and is carried as the bank publishes it."},{"name":"name","type":"string","nullable":true,"desc":"The bank's own label, in Azerbaijani, including the multiplier it prints there — \"1 ABŞ dolları\", \"100 Yapon iyeni\"."},{"name":"unit","type":"string","nullable":false,"desc":"What one of `nominal` is: `unit` for a currency, `troy_ounce` for a metal."},{"name":"nominal","type":"integer","nullable":false,"desc":"How many units `value` is quoted for. Seven currencies quote per 100 — KRW, KZT, HUF, UZS, PKR, RUB and JPY — and everything else per 1. Reading `value` without dividing by this is how a rouble comes out a hundred times too large."},{"name":"value","type":"number","nullable":false,"desc":"The published figure, unrounded: AZN per `nominal` units."},{"name":"rate_per_unit","type":"number","nullable":false,"desc":"`value / nominal`, unrounded. **This is the number you almost certainly want**: AZN for one unit of `code`, or for one troy ounce of a metal."},{"name":"source_url","type":"string","nullable":true,"desc":"The document requested for `date`."},{"name":"bulletin_url","type":"string","nullable":true,"desc":"The document these numbers were published in, i.e. the one for `effective_date`. Differs from `source_url` on a carried-forward row."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]},{"id":"get-rate","verb":"GET","path":"/v1/rates/{id}","summary":"Retrieve a rate","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The rate id from a list response, `\u003csource\u003e:\u003cdate\u003e:\u003ccode\u003e`. Stable for the life of the record: a past bulletin is immutable, so the row this resolves to does not change once it exists.","example":"cbar:2026-09-05:USD"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"Stable identifier, \"\u003csource\u003e:\u003cdate\u003e:\u003ccode\u003e\" — `cbar:2026-09-05:USD`. Opaque: treat the whole string as the id rather than parsing the halves out of it."},{"name":"source","type":"string","nullable":false,"desc":"Which scraper delivered this bulletin. `cbar` is the Central Bank of Azerbaijan."},{"name":"date","type":"date","nullable":false,"desc":"The calendar date this row answers for, **Asia/Baku**. Azerbaijan is UTC+4 year-round with no DST, so a consumer computing \"today\" in UTC gets yesterday's date for the four hours after 20:00 Baku — which is a real bug this feed's predecessor shipped, not a hypothetical one."},{"name":"effective_date","type":"date","nullable":false,"desc":"The date the bank says these rates take effect from — the bulletin's own `ValCurs Date=`, never inferred from the date requested. On a business day it equals `date`; on a weekend, a public holiday or a future date it is the last business day before it."},{"name":"carried_forward","type":"boolean","nullable":false,"desc":"`date != effective_date`: these numbers were published earlier and are still standing. **Filter `carried_forward=false` for a strict business-day series**; take every row for a dense daily one and know which values are repeats. The upstream answers `200` with a full bulletin for every date string including weekends and dates in the future, so this flag is the only thing that distinguishes a fresh publication from a repeat."},{"name":"kind","type":"string","nullable":false,"desc":"`currency` or `metal`. The four bank metals (XAU, XPD, XPT, XAG) are the metals."},{"name":"base","type":"string","nullable":false,"desc":"The currency the rate is expressed in. Always `AZN` for this source."},{"name":"code","type":"string","nullable":false,"desc":"ISO 4217 code, or the metal's X-code. 42 per bulletin: 38 currencies and 4 bank metals. `SDR` is the IMF basket rather than a currency, and is carried as the bank publishes it."},{"name":"name","type":"string","nullable":true,"desc":"The bank's own label, in Azerbaijani, including the multiplier it prints there — \"1 ABŞ dolları\", \"100 Yapon iyeni\"."},{"name":"unit","type":"string","nullable":false,"desc":"What one of `nominal` is: `unit` for a currency, `troy_ounce` for a metal."},{"name":"nominal","type":"integer","nullable":false,"desc":"How many units `value` is quoted for. Seven currencies quote per 100 — KRW, KZT, HUF, UZS, PKR, RUB and JPY — and everything else per 1. Reading `value` without dividing by this is how a rouble comes out a hundred times too large."},{"name":"value","type":"number","nullable":false,"desc":"The published figure, unrounded: AZN per `nominal` units."},{"name":"rate_per_unit","type":"number","nullable":false,"desc":"`value / nominal`, unrounded. **This is the number you almost certainly want**: AZN for one unit of `code`, or for one troy ounce of a metal."},{"name":"source_url","type":"string","nullable":true,"desc":"The document requested for `date`."},{"name":"bulletin_url","type":"string","nullable":true,"desc":"The document these numbers were published in, i.e. the one for `effective_date`. Differs from `source_url` on a carried-forward row."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this version was stored."}]}]},{"slug":"genome","name":"Genome Source Releases API","status":"live","operations":[{"id":"list-genome-releases","verb":"GET","path":"/v1/genome/releases","summary":"List genome source releases","returns":"array","params":[{"name":"status","in":"query","type":"string","required":false,"desc":"How the latest probe went: `ok`, `degraded`, `unreachable` or `error`. `?status=ok` is how you ask for the rows to trust today — a failing source still carries its last good `release_version`.","example":"ok"},{"name":"authority","in":"query","type":"string","required":false,"desc":"The publishing body, exactly as the row carries it — `NCBI`, `EMBL-EBI`. Case-sensitive, and an unknown value answers an empty page rather than a 400, because this vocabulary is the upstream registry's and grows when a source from a new body is added.","example":"NCBI"},{"name":"category","in":"query","type":"string","required":false,"desc":"The registry's own kind label — `reference_sequence`, `genome_metadata`, `variation`, `population_reference`. Open for the same reason `authority` is.","example":"variation"},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, each optionally prefixed with `-` to reverse. Default is `-ingested_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-ingested_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"The upstream registry's own key — `refseq`, `dbsnp`, `igsr_1000g`, `ncbi_datasets`. **The one id on this API with no `\u003csource\u003e:` prefix**, because the row's identity IS an upstream source; it is the same string as `source_id` and it is what `/v1/genome/releases/{id}` takes."},{"name":"source","type":"string","nullable":false,"desc":"Which softon scraper delivered this row. One probe pass reads every source here, so it is the same value on every row — `source_id` is the field that varies, and the one you almost certainly mean."},{"name":"source_id","type":"string","nullable":false,"desc":"The upstream registry's key, identical to `id`. Published under this name because it is the name the registry this replaces uses, so a consumer writing softon's answer back into that registry keys on it without a mapper."},{"name":"source_name","type":"string","nullable":false,"desc":"What the source calls itself: `RefSeq`, `dbSNP`, `1000 Genomes / IGSR`, `NCBI Datasets`."},{"name":"authority","type":"string","nullable":false,"desc":"The body that publishes it — `NCBI` or `EMBL-EBI`."},{"name":"category","type":"string","nullable":false,"desc":"The registry's own label for what kind of source this is: `reference_sequence`, `genome_metadata`, `variation`, `population_reference`. A shared free-form vocabulary rather than a closed set, so `?category=` accepts anything and answers an empty page for a label no source carries."},{"name":"landing_url","type":"string","nullable":true,"desc":"The human-facing page for this source. Never the FTP index — this is the one to link a reader to."},{"name":"api_url","type":"string","nullable":true,"desc":"The source's machine-facing API root, where it has one. Null for RefSeq and IGSR, which have none."},{"name":"bulk_url","type":"string","nullable":true,"desc":"The source's bulk archive root, where it has one. Null for NCBI Datasets. **Recorded, never mirrored**: nothing on this platform downloads it, and this API serves no data from behind it."},{"name":"update_frequency","type":"string","nullable":false,"desc":"How often the SOURCE cuts a release — `periodic_release`, `release_based`, `continuous`. Not how often softon probes it: the probe runs daily whatever this says."},{"name":"release_version","type":"string","nullable":true,"desc":"The release the source is currently on, **as a string and never a number**: `\"237\"`, `\"18.37.0\"`, `\"157\"`, `\"20130502\"`. Release names are not all numeric and a JSON number would also defeat a `?? \"—\"` null check the day one of them is `0`.\n\n**This is a release MARKER and nothing else.** It pins no download, selects no file and carries no sequence, assembly, annotation or variant data — for RefSeq, for instance, assembly facts come from the separate ASSEMBLY_REPORTS bulk tables and not from anything here.\n\nNull until a probe has succeeded once. After that it is **sticky**: a probe that fails leaves the last good value standing and moves `status` instead, so read the two together rather than treating a non-null version as fresh."},{"name":"release_date","type":"date","nullable":true,"desc":"The release's own date where the source publishes one. Today only `igsr_1000g` has it, and there it is the **freeze name** sliced out of a directory name (`20130502` → `2013-05-02`) — an identity, not a publication date. Deliberately not filled from an upstream `Last-Modified`, which is a different fact."},{"name":"checksum","type":"string","nullable":true,"desc":"The version signal for sources identified by a digest rather than a number. Null on every source served today."},{"name":"format","type":"string","nullable":true,"desc":"How the UPSTREAM distributes its data — `flatfile`, `rest_v2`, `ftp_release_dirs`, `text_release_notes`. Not the format of this response, which is JSON for all of them."},{"name":"retrieved_at","type":"timestamp","nullable":true,"desc":"When softon last **successfully** read this upstream. Not when your request arrived, and never advanced by a probe that failed — it is a provenance claim, and a failed refresh that moved it would assert a read that did not happen. Compare it with `ingested_at`, which is the last probe of any kind, to see how stale the answer is. Null only for a source that has never answered."},{"name":"status","type":"string","nullable":false,"desc":"How the LATEST probe went. Four values, and they are four because the distinctions matter to different people:\n\n`ok` — a release was read. `degraded` — the host answered 2xx and nothing parseable came back, so the upstream's format moved and a person has to fix the probe. `unreachable` — the host never answered: DNS, TLS, connection or timeout. Nothing is wrong with softon's parser and nothing is wrong with the data. `error` — the host answered with a non-2xx.\n\nThe feed this replaces collapsed `unreachable` into `degraded` and then counted the source as reachable anyway, so an outage read as a parser bug."},{"name":"extra","type":"object","nullable":true,"desc":"Per-source probe metadata: which URL was read, what it answered, and the handful of fields the upstream published beside the version. **An allow-listed projection, not the upstream body** — one of these endpoints echoes a submitted API key back inside its own error body, so a verbatim pass-through would publish a credential. Null where a source records none."},{"name":"sort_order","type":"integer","nullable":false,"desc":"This source's position in the upstream registry's declaration order, 1–10. Sort on it to reproduce that order: this collection pages on `ingested_at` like every other one here, and one probe pass stores every row at the same instant, so the order rows come back in is the id's and not the registry's."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When the latest probe ran and this version was stored — successful or not. See `retrieved_at` for the last successful read."}]},{"id":"get-genome-release","verb":"GET","path":"/v1/genome/releases/{id}","summary":"Retrieve one genome source release","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The upstream registry key, e.g. `refseq`. **No `\u003csource\u003e:` prefix on this collection** — the id of a release marker is the source it describes, and it is stable for the life of the source.","example":"refseq"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"The upstream registry's own key — `refseq`, `dbsnp`, `igsr_1000g`, `ncbi_datasets`. **The one id on this API with no `\u003csource\u003e:` prefix**, because the row's identity IS an upstream source; it is the same string as `source_id` and it is what `/v1/genome/releases/{id}` takes."},{"name":"source","type":"string","nullable":false,"desc":"Which softon scraper delivered this row. One probe pass reads every source here, so it is the same value on every row — `source_id` is the field that varies, and the one you almost certainly mean."},{"name":"source_id","type":"string","nullable":false,"desc":"The upstream registry's key, identical to `id`. Published under this name because it is the name the registry this replaces uses, so a consumer writing softon's answer back into that registry keys on it without a mapper."},{"name":"source_name","type":"string","nullable":false,"desc":"What the source calls itself: `RefSeq`, `dbSNP`, `1000 Genomes / IGSR`, `NCBI Datasets`."},{"name":"authority","type":"string","nullable":false,"desc":"The body that publishes it — `NCBI` or `EMBL-EBI`."},{"name":"category","type":"string","nullable":false,"desc":"The registry's own label for what kind of source this is: `reference_sequence`, `genome_metadata`, `variation`, `population_reference`. A shared free-form vocabulary rather than a closed set, so `?category=` accepts anything and answers an empty page for a label no source carries."},{"name":"landing_url","type":"string","nullable":true,"desc":"The human-facing page for this source. Never the FTP index — this is the one to link a reader to."},{"name":"api_url","type":"string","nullable":true,"desc":"The source's machine-facing API root, where it has one. Null for RefSeq and IGSR, which have none."},{"name":"bulk_url","type":"string","nullable":true,"desc":"The source's bulk archive root, where it has one. Null for NCBI Datasets. **Recorded, never mirrored**: nothing on this platform downloads it, and this API serves no data from behind it."},{"name":"update_frequency","type":"string","nullable":false,"desc":"How often the SOURCE cuts a release — `periodic_release`, `release_based`, `continuous`. Not how often softon probes it: the probe runs daily whatever this says."},{"name":"release_version","type":"string","nullable":true,"desc":"The release the source is currently on, **as a string and never a number**: `\"237\"`, `\"18.37.0\"`, `\"157\"`, `\"20130502\"`. Release names are not all numeric and a JSON number would also defeat a `?? \"—\"` null check the day one of them is `0`.\n\n**This is a release MARKER and nothing else.** It pins no download, selects no file and carries no sequence, assembly, annotation or variant data — for RefSeq, for instance, assembly facts come from the separate ASSEMBLY_REPORTS bulk tables and not from anything here.\n\nNull until a probe has succeeded once. After that it is **sticky**: a probe that fails leaves the last good value standing and moves `status` instead, so read the two together rather than treating a non-null version as fresh."},{"name":"release_date","type":"date","nullable":true,"desc":"The release's own date where the source publishes one. Today only `igsr_1000g` has it, and there it is the **freeze name** sliced out of a directory name (`20130502` → `2013-05-02`) — an identity, not a publication date. Deliberately not filled from an upstream `Last-Modified`, which is a different fact."},{"name":"checksum","type":"string","nullable":true,"desc":"The version signal for sources identified by a digest rather than a number. Null on every source served today."},{"name":"format","type":"string","nullable":true,"desc":"How the UPSTREAM distributes its data — `flatfile`, `rest_v2`, `ftp_release_dirs`, `text_release_notes`. Not the format of this response, which is JSON for all of them."},{"name":"retrieved_at","type":"timestamp","nullable":true,"desc":"When softon last **successfully** read this upstream. Not when your request arrived, and never advanced by a probe that failed — it is a provenance claim, and a failed refresh that moved it would assert a read that did not happen. Compare it with `ingested_at`, which is the last probe of any kind, to see how stale the answer is. Null only for a source that has never answered."},{"name":"status","type":"string","nullable":false,"desc":"How the LATEST probe went. Four values, and they are four because the distinctions matter to different people:\n\n`ok` — a release was read. `degraded` — the host answered 2xx and nothing parseable came back, so the upstream's format moved and a person has to fix the probe. `unreachable` — the host never answered: DNS, TLS, connection or timeout. Nothing is wrong with softon's parser and nothing is wrong with the data. `error` — the host answered with a non-2xx.\n\nThe feed this replaces collapsed `unreachable` into `degraded` and then counted the source as reachable anyway, so an outage read as a parser bug."},{"name":"extra","type":"object","nullable":true,"desc":"Per-source probe metadata: which URL was read, what it answered, and the handful of fields the upstream published beside the version. **An allow-listed projection, not the upstream body** — one of these endpoints echoes a submitted API key back inside its own error body, so a verbatim pass-through would publish a credential. Null where a source records none."},{"name":"sort_order","type":"integer","nullable":false,"desc":"This source's position in the upstream registry's declaration order, 1–10. Sort on it to reproduce that order: this collection pages on `ingested_at` like every other one here, and one probe pass stores every row at the same instant, so the order rows come back in is the id's and not the registry's."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When the latest probe ran and this version was stored — successful or not. See `retrieved_at` for the last successful read."}]}]},{"slug":"reachability","name":"Reachability API","status":"live","operations":[{"id":"list-reachability","verb":"GET","path":"/v1/reachability","summary":"List watched targets and how each one last answered","returns":"array","params":[{"name":"watchlist","in":"query","type":"string","required":false,"desc":"The named set of targets to read — `middle-corridor` is the first one. Open rather than a closed set, because registering a watchlist is a row rather than a release, so an unknown value answers an empty page rather than a 400.","example":"middle-corridor"},{"name":"vantage","in":"query","type":"string","required":false,"desc":"Where the probe was made from, e.g. `hetzner-nbg1`. Narrowing on it is how you compare two vantages' answers for the same target — which is the comparison this collection exists to make routine.","example":"hetzner-nbg1"},{"name":"reachable","in":"query","type":"boolean","required":false,"desc":"`true` for targets whose last probe got a 200, `false` for the rest. Omit it for both. **`reachable=false` is not a list of sites that are down** — pair it with `failure_class` before drawing that conclusion.","example":"false"},{"name":"failure_class","in":"query","type":"string","required":false,"desc":"One of `http_error`, `blocked_by_bot_protection`, `tls_error`, `dns_error`, `timeout`, `connection_error`, `robots_disallowed`. Closed, unlike `watchlist` and `vantage`, because this one is our own judgement and a typo in it would mean you believed you had filtered. `?failure_class=blocked_by_bot_protection` is how you ask which of these are our problem rather than theirs.","example":"blocked_by_bot_protection"},{"name":"limit","in":"query","type":"integer","required":false,"desc":"Items per page, 1–100. Defaults to 25.","example":"50"},{"name":"cursor","in":"query","type":"string","required":false,"desc":"Opaque cursor from a previous response's meta.next. Do not construct one."},{"name":"order","in":"query","type":"string","required":false,"desc":"Sort field: `ingested_at`, each optionally prefixed with `-` to reverse. Default is `-ingested_at`. Rows with no value for the sort column always sort LAST in either direction, so paging never leads with undated rows.","example":"-ingested_at"},{"name":"source","in":"query","type":"string","required":false,"desc":"Restrict to ONE scraper source, by its stem — `abb`, not `ABB` or `abb-bank.az`. Not a list: `?source=a,b` and a repeated `?source=` do not select two sources. On `/v1/jobs` an unknown stem is a `400` naming it, and `source_type` is the way to select several sources at once. `GET /v1/sources` lists every stem."}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"`\u003csource\u003e:\u003ctarget_id\u003e` — the prober and the target it asked. Stable for the life of the target, and what `/v1/reachability/{id}` takes."},{"name":"source","type":"string","nullable":false,"desc":"Which prober made the request. **This is the one collection here where `source` is softon and not an upstream publisher**: a row is a record of a request this platform made, so the source is the client that made it. A second prober at a second vantage is a second source, and a second row for the same target."},{"name":"target_id","type":"string","nullable":false,"desc":"The stable key for one watched URL. Chosen when the target is registered and never derived from the URL, so a target that changes host keeps the history in the three fields at the bottom of this table."},{"name":"url","type":"string","nullable":false,"desc":"The URL that was requested, exactly as registered."},{"name":"watchlist","type":"string","nullable":false,"desc":"The named set of targets this one belongs to, and the value `?watchlist=` takes. `middle-corridor` is the first one. Open rather than a closed set — a new watchlist is a row, not a release — so an unknown value answers an empty page rather than a 400."},{"name":"vantage","type":"string","nullable":false,"desc":"**Where the request was made from**, and the field that stops this row being read as a verdict on the site. `hetzner-nbg1` is a datacenter address in Nuremberg, which is this platform's ingestion box.\n\nThe answer genuinely depends on where you ask from: the same URL has been measured answering `403` to a datacenter address and `200` to a residential one on the same day. A reachability figure quoted without the vantage attached is a measurement of a pair reported as a property of one half of it — which is also why the prober is carried as `source`, so that a second client probing the same target is a second row rather than a silent redefinition of this one."},{"name":"reachable","type":"boolean","nullable":false,"desc":"Whether the last probe got an HTTP **200**. Strictly 200: a `204` or a `206` reads as false here, deliberately, because that is what the pipeline this replaces means by the word.\n\n**`false` is not a claim that the site is down.** Two of the six targets served today sit behind a bot-protection edge whose answer is decided by the request rather than by the site: measured with one client, one user agent, one address and one minute, both answered `403` on every HTTP/2 sample and `200` on every HTTP/1.1 one, and one of them is not stable even within a single client.\n\nSo read `failure_class` before turning this into a count, and `consecutive_failures` before turning one row into a finding. A single `blocked_by_bot_protection` is noise; sixty-one consecutive ones are a fact about a relationship with that site."},{"name":"http_status","type":"integer","nullable":true,"desc":"The final status after redirects. Null when no HTTP response was obtained at all: a name that did not resolve, a handshake that was refused, a timeout, or a target whose robots.txt means we did not ask. Always present as a key, null where inapplicable."},{"name":"final_url","type":"string","nullable":true,"desc":"Where the request ended up after redirects. Null when there was no response."},{"name":"redirected","type":"boolean","nullable":false,"desc":"Whether the request moved before answering. False when there was no response."},{"name":"failure_class","type":"string","nullable":true,"desc":"Why it was not a 200, from a closed set, and **null exactly when `reachable` is true** — the database refuses either half without the other, so there is no such row as one that failed without saying why.\n\n`http_error` — the host answered, with something that is not 200. `blocked_by_bot_protection` — the host answered 403 with a bot-protection challenge page. **Our client failed a check; the site is up.** It is separate from `http_error` because it is a fact about the request rather than about the target, and separate from an outage because collapsing the two publishes a finding about a site that is serving fine. `tls_error` — the certificate or the handshake was refused; a browser would refuse it too. `dns_error` — the name did not resolve. `timeout` — nothing came back inside the budget. `connection_error` — the connection failed or was reset below HTTP. `robots_disallowed` — **we did not ask.** The target's robots.txt refuses this crawler for this path, so `http_status` is null because no request was made, not because one failed."},{"name":"error","type":"string","nullable":true,"desc":"A short plain sentence, at most 200 characters, or null. `\"TLS certificate expired\"`, not a stack trace.\n\n**Treat it as user-facing copy**, because on the page this dataset was built for it is rendered verbatim to the public. The pipeline it replaces writes `str(exception)` into the equivalent field, which is how a visitor to a transport-analysis page came to read a urllib3 connection-pool repr complete with a CPython source-file line number.\n\nNull rather than truncated when a sentence cannot be vouched for: a shortened exception repr is still an exception repr, and the meaning is in `failure_class`, which is never null on a failing row."},{"name":"response_ms","type":"integer","nullable":true,"desc":"How long the request took, in milliseconds. Null when nothing came back. One measurement from one vantage on a shared box — useful for noticing a target getting slower, not a latency SLA."},{"name":"robots_allowed","type":"boolean","nullable":false,"desc":"What the target's own `robots.txt` says about this crawler fetching this path. Nothing in the pipeline this replaces checks robots at all.\n\nFails **open** when robots.txt cannot be fetched — a 404 or a timeout there is not a refusal. When it is false no request is made, and `failure_class` is `robots_disallowed`."},{"name":"checked_at","type":"timestamp","nullable":false,"desc":"When the probe ran, on the prober's clock. **Date a finding by this**, not by `ingested_at`: the pipeline this replaces stamps its rows with the date of the build that read them, so a row could claim it was checked on a day nothing was."},{"name":"last_reachable_at","type":"timestamp","nullable":true,"desc":"The last probe that got a 200. Null for a target that has never answered one."},{"name":"first_failed_at","type":"timestamp","nullable":true,"desc":"When the current run of failures began. Null while the target is reachable, and reset the moment it answers 200 again."},{"name":"consecutive_failures","type":"integer","nullable":false,"desc":"How many probes in a row have not got a 200; 0 while the target is reachable.\n\nThis and the two fields above are the value this collection has that a one-shot check cannot: \"this target has been refusing us for sixty-one straight probes\" is an editorial signal, and \"it refused us once, today\" is not. They are accumulated where the row is stored, so they survive a redeploy and the prober never has to remember anything.\n\nCounts **probes**, not days — the same number at the daily cadence this runs at, and named for the fact so that changing the cadence cannot silently redefine it. All three reset if `vantage` changes, because a counter spanning two vantages answers neither question."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this platform stored the observation. Differs from `checked_at` by the time the batch spent in flight; compare the two to see whether probe runs are landing at all."}]},{"id":"get-reachability","verb":"GET","path":"/v1/reachability/{id}","summary":"Retrieve one watched target","returns":"object","params":[{"name":"id","in":"path","type":"string","required":true,"desc":"The target id from a list response, `\u003csource\u003e:\u003ctarget_id\u003e`. Stable for the life of the target: the id survives the target changing host, which is what keeps its failure history attached to it.","example":"softonbot:az-transit-gov"}],"fields":[{"name":"id","type":"string","nullable":false,"desc":"`\u003csource\u003e:\u003ctarget_id\u003e` — the prober and the target it asked. Stable for the life of the target, and what `/v1/reachability/{id}` takes."},{"name":"source","type":"string","nullable":false,"desc":"Which prober made the request. **This is the one collection here where `source` is softon and not an upstream publisher**: a row is a record of a request this platform made, so the source is the client that made it. A second prober at a second vantage is a second source, and a second row for the same target."},{"name":"target_id","type":"string","nullable":false,"desc":"The stable key for one watched URL. Chosen when the target is registered and never derived from the URL, so a target that changes host keeps the history in the three fields at the bottom of this table."},{"name":"url","type":"string","nullable":false,"desc":"The URL that was requested, exactly as registered."},{"name":"watchlist","type":"string","nullable":false,"desc":"The named set of targets this one belongs to, and the value `?watchlist=` takes. `middle-corridor` is the first one. Open rather than a closed set — a new watchlist is a row, not a release — so an unknown value answers an empty page rather than a 400."},{"name":"vantage","type":"string","nullable":false,"desc":"**Where the request was made from**, and the field that stops this row being read as a verdict on the site. `hetzner-nbg1` is a datacenter address in Nuremberg, which is this platform's ingestion box.\n\nThe answer genuinely depends on where you ask from: the same URL has been measured answering `403` to a datacenter address and `200` to a residential one on the same day. A reachability figure quoted without the vantage attached is a measurement of a pair reported as a property of one half of it — which is also why the prober is carried as `source`, so that a second client probing the same target is a second row rather than a silent redefinition of this one."},{"name":"reachable","type":"boolean","nullable":false,"desc":"Whether the last probe got an HTTP **200**. Strictly 200: a `204` or a `206` reads as false here, deliberately, because that is what the pipeline this replaces means by the word.\n\n**`false` is not a claim that the site is down.** Two of the six targets served today sit behind a bot-protection edge whose answer is decided by the request rather than by the site: measured with one client, one user agent, one address and one minute, both answered `403` on every HTTP/2 sample and `200` on every HTTP/1.1 one, and one of them is not stable even within a single client.\n\nSo read `failure_class` before turning this into a count, and `consecutive_failures` before turning one row into a finding. A single `blocked_by_bot_protection` is noise; sixty-one consecutive ones are a fact about a relationship with that site."},{"name":"http_status","type":"integer","nullable":true,"desc":"The final status after redirects. Null when no HTTP response was obtained at all: a name that did not resolve, a handshake that was refused, a timeout, or a target whose robots.txt means we did not ask. Always present as a key, null where inapplicable."},{"name":"final_url","type":"string","nullable":true,"desc":"Where the request ended up after redirects. Null when there was no response."},{"name":"redirected","type":"boolean","nullable":false,"desc":"Whether the request moved before answering. False when there was no response."},{"name":"failure_class","type":"string","nullable":true,"desc":"Why it was not a 200, from a closed set, and **null exactly when `reachable` is true** — the database refuses either half without the other, so there is no such row as one that failed without saying why.\n\n`http_error` — the host answered, with something that is not 200. `blocked_by_bot_protection` — the host answered 403 with a bot-protection challenge page. **Our client failed a check; the site is up.** It is separate from `http_error` because it is a fact about the request rather than about the target, and separate from an outage because collapsing the two publishes a finding about a site that is serving fine. `tls_error` — the certificate or the handshake was refused; a browser would refuse it too. `dns_error` — the name did not resolve. `timeout` — nothing came back inside the budget. `connection_error` — the connection failed or was reset below HTTP. `robots_disallowed` — **we did not ask.** The target's robots.txt refuses this crawler for this path, so `http_status` is null because no request was made, not because one failed."},{"name":"error","type":"string","nullable":true,"desc":"A short plain sentence, at most 200 characters, or null. `\"TLS certificate expired\"`, not a stack trace.\n\n**Treat it as user-facing copy**, because on the page this dataset was built for it is rendered verbatim to the public. The pipeline it replaces writes `str(exception)` into the equivalent field, which is how a visitor to a transport-analysis page came to read a urllib3 connection-pool repr complete with a CPython source-file line number.\n\nNull rather than truncated when a sentence cannot be vouched for: a shortened exception repr is still an exception repr, and the meaning is in `failure_class`, which is never null on a failing row."},{"name":"response_ms","type":"integer","nullable":true,"desc":"How long the request took, in milliseconds. Null when nothing came back. One measurement from one vantage on a shared box — useful for noticing a target getting slower, not a latency SLA."},{"name":"robots_allowed","type":"boolean","nullable":false,"desc":"What the target's own `robots.txt` says about this crawler fetching this path. Nothing in the pipeline this replaces checks robots at all.\n\nFails **open** when robots.txt cannot be fetched — a 404 or a timeout there is not a refusal. When it is false no request is made, and `failure_class` is `robots_disallowed`."},{"name":"checked_at","type":"timestamp","nullable":false,"desc":"When the probe ran, on the prober's clock. **Date a finding by this**, not by `ingested_at`: the pipeline this replaces stamps its rows with the date of the build that read them, so a row could claim it was checked on a day nothing was."},{"name":"last_reachable_at","type":"timestamp","nullable":true,"desc":"The last probe that got a 200. Null for a target that has never answered one."},{"name":"first_failed_at","type":"timestamp","nullable":true,"desc":"When the current run of failures began. Null while the target is reachable, and reset the moment it answers 200 again."},{"name":"consecutive_failures","type":"integer","nullable":false,"desc":"How many probes in a row have not got a 200; 0 while the target is reachable.\n\nThis and the two fields above are the value this collection has that a one-shot check cannot: \"this target has been refusing us for sixty-one straight probes\" is an editorial signal, and \"it refused us once, today\" is not. They are accumulated where the row is stored, so they survive a redeploy and the prober never has to remember anything.\n\nCounts **probes**, not days — the same number at the daily cadence this runs at, and named for the fact so that changing the cadence cannot silently redefine it. All three reset if `vantage` changes, because a counter spanning two vantages answers neither question."},{"name":"ingested_at","type":"timestamp","nullable":false,"desc":"When this platform stored the observation. Differs from `checked_at` by the time the batch spent in flight; compare the two to see whether probe runs are landing at all."}]}]}],"shared":[{"id":"list-sources","verb":"GET","path":"/v1/sources","summary":"Every source feeding every dataset, with how much it holds and when it last delivered","returns":"array","params":[],"fields":[{"name":"source","type":"string","nullable":false,"desc":"The stem, and the value `?source=` takes — `abb`, not `ABB` or `abb-bank.az`. It is also the first half of every posting `id`."},{"name":"name","type":"string","nullable":true,"desc":"What the upstream calls itself (`1is.az`, `ABB`). Null for a source with no registry entry — the events sources have none."},{"name":"url","type":"string","nullable":true,"desc":"The source's own website. The same value `source_url` carries on a job row, and set for the same 122 of 130. Null on eight rather than guessed: three live entirely on a third party's ATS, one has only its ATS vendor's domain recorded, `other` is a catch-all, and three we have measured as gone. Also null on the events, quotes and rates rows, which have no registry entry at all — the same reason `name` is null there."},{"name":"jobs_url","type":"string","nullable":true,"desc":"The page this platform reads this source's vacancies from — `https://jobs.glorri.az` for `glorri`, an employer's careers page, or the candidate-facing page on the ATS for an employer hosted on one. **This is the one to link a human to.**\n\nRead off the scraper rather than curated, so unlike `source_type` it is a measurement and not an opinion. Set for 106 sources; null for 24 — nineteen whose scraper was archived, whose rows are still in the corpus and whose site is often still up but whose listing page is recorded nowhere; two that only ever arrived through a history import; and three whose page we have measured as gone."},{"name":"dataset","type":"string","nullable":false,"desc":"Which collection this source feeds: `jobs`, `events`, `quotes`, `rates`, `genome` or `reachability`."},{"name":"source_type","type":"string","nullable":true,"desc":"What KIND of publisher this is — `employer`, `platform`, `agency`, `board`, `aggregator` or `unknown` — and the value `/v1/jobs?source_type=` takes. **This is our classification, not a fact the source asserts**: it is the one editorial field in this API, a value can change when a source is re-read, and the SET of values is stable. Null on the events, quotes, rates, genome and reachability rows, because the distinction is about who publishes a VACANCY and means nothing applied to a ticketing site, a quotations corpus, a central bank, a sequence archive, or this platform's own probe."},{"name":"records","type":"integer","nullable":false,"desc":"How many rows of this source are in the current-state table — what `/v1/jobs?source=\u003cthis\u003e` returns with `?active=` left at its default of `any`. **It counts postings the source has already taken down**, and for jobs most of them are: 149,383 against 39,046 still open when this was written. Two sources report thousands here and return nothing at all, their scrapers having stopped. If you are sizing live vacancies, read `live_records` instead. It also does not apply the suppression anti-join, which is harmless while nothing is suppressed and wrong the day something is."},{"name":"live_records","type":"integer","nullable":false,"desc":"How many postings of this source are STILL OPEN — the number `/v1/jobs?source=\u003cthis\u003e\u0026active=true` returns, counted the same way the read API counts it. **This is the number to plan against.** For events, quotes, rates, genome and reachability it equals `records`: an event is not retired, it simply passes, the quotes view is already active-only, a published exchange rate for a past date is a fact rather than a listing that can close, and a release marker or a watched target is a current state rather than a row that expires — one whose probe is failing is still counted, which is what `status` and `failure_class` on those rows are for."},{"name":"aggregator","type":"boolean","nullable":false,"desc":"True when this source republishes OTHER boards' postings under its own id, with the ORIGINAL board's url. Two records then exist for one posting — same `url`, different `source`, different `id` — so a consumer keying a table on `url` will collide its copy with the owning board's record."},{"name":"last_ingested_at","type":"string","nullable":true,"desc":"When a batch from this source was last accepted, RFC 3339. It says the scraper ran, NOT that anything changed — and a source quiet for months still answers with its rows. Null only if no batch was ever recorded."},{"name":"quality","type":"object","nullable":true,"desc":"How USABLE this source's live rows are, as opposed to how many there are. Three whole percentages over the rows `?source=\u003cthis\u003e\u0026active=true` returns: `description_pct`, `location_pct` and `duplicate_url_pct`.\n\n`records` and `live_records` say how much is there and nothing about whether it can be used. Two sources publish rows with no body at all — one carries a title, a company and a date and nothing else — so a consumer filtering for postings with a description finds those rows contribute zero, after building the integration rather than before.\n\n`duplicate_url_pct` is the same-source arity `url` already documents — one vacancy under two of the board's own ids — and the advice there still holds: collapse on the newer `ingested_at`. What this adds is the RATE, because a quirk affecting a handful of rows and one affecting a quarter of a source are different facts if you are sizing a mirror. Cross-source duplication is a different problem and `aggregator` marks it.\n\nNull on events, quotes, rates, genome and reachability, and on a jobs source with no live rows left to measure — which is not the same as a source whose rows all score zero."}]}],"envelope":[{"name":"data","type":"array | object","nullable":false,"desc":"The result. An array for collections, one object for a single resource."},{"name":"meta","type":"object","nullable":false,"desc":"Pagination and request metadata. Always present."},{"name":"meta.next","type":"string","nullable":true,"desc":"Opaque cursor for the following page. null on the last page."},{"name":"meta.count","type":"integer","nullable":true,"desc":"Items in data. Present for collections only."},{"name":"meta.total","type":"integer","nullable":true,"desc":"Rows the FILTER matches across every page, not just this one. Present only when you ask with `?count=true`, because it costs a second query."},{"name":"meta.requested","type":"integer","nullable":true,"desc":"Ids you sent. Batch endpoints only."},{"name":"meta.answered","type":"integer","nullable":true,"desc":"Rows in data. Batch endpoints only, and equal to `requested` by contract — a batch either answers every id or fails."},{"name":"meta.request_id","type":"string","nullable":false,"desc":"Quote this when reporting a problem."},{"name":"error","type":"object","nullable":true,"desc":"null on success. On failure, carries code and message."}],"errors":[{"status":400,"code":"invalid_request","meaning":"A parameter is missing, malformed or out of range. The message names it.","retry":false},{"status":401,"code":"unauthorized","meaning":"The key is missing, malformed or revoked.","retry":false},{"status":403,"code":"forbidden","meaning":"The key is valid but your plan does not include this dataset.","retry":false},{"status":404,"code":"not_found","meaning":"No resource with that id on this dataset.","retry":false},{"status":410,"code":"gone","meaning":"This record was held and has since been removed at the request of the employer or the individual concerned. It will not come back. If you hold a copy, stop serving it. Returned only by the single-resource paths, and only for an id this platform actually had -- an id that never existed here is still a 404.","retry":false},{"status":429,"code":"rate_limited","meaning":"Plan rate exceeded. Retry-After gives the wait in seconds.","retry":true},{"status":429,"code":"quota_exceeded","meaning":"The account's monthly request allowance is spent. It resets on the 1st; X-Quota-Limit and X-Quota-Used say where you are. Retrying sooner cannot help, which is why this is a different code from rate_limited.","retry":false}],"limits":{"default_page_size":25,"max_page_size":100,"plans":[{"name":"free","per_second":5,"burst":10,"monthly_requests":1000,"datasets":["jobs","events","quotes","rates","genome","reachability"]},{"name":"starter","per_second":10,"burst":20,"monthly_requests":5000,"datasets":["jobs","events","quotes","rates","genome","reachability"]},{"name":"builder","per_second":50,"burst":100,"monthly_requests":50000,"datasets":["jobs","events","quotes","rates","genome","reachability"]},{"name":"scale","per_second":100,"burst":200,"monthly_requests":250000,"datasets":["jobs","events","quotes","rates","genome","reachability"]}],"caching":{"etag":true,"if_none_match":true,"not_modified_status":304,"not_modified_is_billed":false,"cache_control":"private, max-age=0, must-revalidate","notes":"Every 200 on a collection or a single resource carries a strong ETag over the DATA — not the envelope, so the per-request `request_id` does not change it. Send it back as `If-None-Match` and an unchanged page answers 304 with no body. A 304 costs no credit, because a credit is one successful (2xx) request and 304 is not one; the per-second rate limit still applies. `private` because a response is scoped to your key — do not put these in a shared cache. For tailing new and changed rows, `updated_after` is still cheaper than any revalidation; conditional requests are for periodically verifying a FULL set."}}},"meta":{"next":null,"request_id":"req_145a57e727acf907"},"error":null}