Israel's open data portal has no changelog, so I built one

A few weeks ago I went looking for a particular dataset on data.gov.il, Israel's national open data portal. I found it, which was the easy part. The harder question arrived immediately afterwards: what else has appeared here since the last time I looked?

There is no way to ask. The portal gives you a search box, a list of publishing bodies, and just under 1,200 datasets (1,197 on the day I'm writing this). What it does not give you is a changelog — no "recently added" view, no feed, no "new this month" page. When the Ministry of Transport published a dataset of metro depot locations on the 1st of June, the only way to find out was to already be looking.

That gap nagged at me, mostly because open data portals exist for people who want to build things, and people who want to build things need to know when the raw material changes. A catalogue you can only browse is a library with no new-acquisitions shelf.

So I built the shelf. Israel-Open-Data-Changelog is 1,197 datasets sorted into 124 monthly folders, from December 2015 to this month, each month a README table and a CSV. It is entirely unofficial — nothing to do with data.gov.il or the Israeli government, just a third party poking at a public API.

CKAN, briefly

All of this runs on one piece of software, and it's worth a paragraph, because it explains why the reconstruction was possible at all.

CKAN — Comprehensive Knowledge Archive Network, a name nobody uses — is an open-source data catalogue built by the Open Knowledge Foundation. It is the default answer to "our government has decided to publish open data, now what?", and it runs a large share of the world's national portals: data.gov.uk, the European data portal, various Canadian, Australian and Brazilian equivalents, and data.gov.il (which is part of why so many of them look oddly alike). The important thing about CKAN here is that it is a catalogue rather than a database. It doesn't really store your CSVs; it stores records about your CSVs — a title, a description, a publishing organisation, some tags, and links to the actual files, which CKAN calls resources.

And it exposes all of that over a JSON API that is open by default. Every CKAN instance has an endpoint at /api/3/action/, and the actions carry the same names everywhere: package_search, package_show, organization_list. Learn one CKAN and you have learned all of them, which is an underrated piece of standardisation.

How the table is derived

Here is the thing that makes a changelog possible at all: every CKAN record carries a metadata_created timestamp, and package_search will sort by it.

GET https://data.gov.il/api/3/action/package_search
      ?rows=1000
      &start=0
      &sort=metadata_created%20desc

That is the whole trick. One endpoint, one sort parameter, two requests to walk the entire catalogue (rows caps out at 1,000). Nothing is scraped, no dates are inferred, nothing is guessed. Each dataset drops into the month its own timestamp names, the tree of monthly folders is regenerated from scratch on every run, and the build is idempotent — so a run against an unchanged portal produces no diff whatsoever. Which means git diff after a build is a literal answer to "what did the portal add since last time".

Two API quirks that cost me time and may save you some. The response echoes back "sort": "metadata_created desc", which is how you confirm the parameter was honoured rather than quietly ignored (a small kindness, and not one every API extends). And the fl field-list parameter is not supported — it returns an HTTP 500 rather than a well-behaved 400, so if a request suddenly starts failing, suspect an unsupported parameter before you suspect an outage. Two smaller ones: status_show returns 403 on this portal while the rest of the action API is wide open, and organization_list reports null package counts, so publisher tallies have to be computed from search results instead.

What actually landed recently

I pulled the live API again while writing this, on the 27th of July 2026. The most recent arrivals:

• 15 July — Traffic Courts by District, Small Claims Courts by District and Family Courts by District, all three from the Judicial Authority. Each one takes a locality name and returns the court with jurisdiction over it.

• 12 July — Grants Report for Calls for Proposals, Local Government Division, from the Ministry of Energy's sustainable energy unit. Grants for promoting renewable energy in local government.

• 23 and 21 June — two entries in a Data Dictionary Library from the National Digital Agency, standardising terminology for business-entity and asset-entity data.

• 1 June — Tunnels and Metro Depots, both from the Ministry of Transport and Road Safety.

Four datasets in July, four in June, six in May. That is roughly the current run rate, and it is a long way down from the portal's earlier years — 181 datasets in 2017, 134 in 2018, 131 in 2019, against 50 across the whole of 2025. Make of that what you will. (I have a suspicion, but a suspicion is all it is, and one reason to keep this thing running is the hope of eventually replacing it with something better.)

The two visible lumps in the timeline are genuine portal history rather than artefacts: 113 datasets in December 2015, which is the launch seed load, and 91 in November 2017, a second onboarding wave.

What the dates are not

This is the load-bearing caveat, and I checked it before building anything, because government CKAN instances have a well-earned habit of bulk-importing their entire catalogue on migration day and flattening the real publication history in the process.

data.gov.il behaves well. All 1,197 datasets have a metadata_created value — none missing, which is rarer than it ought to be. The timestamps are microsecond-resolution and land minutes apart within a session: the three Judicial Authority datasets above were stamped at 12:24, 12:27 and 12:29 on the same afternoon, which reads as three real publication events rather than one batch job. And the additions spread across 124 distinct months out of a possible 128.

But it is still the record's creation timestamp in the portal's own database. Nobody at data.gov.il documents it as a publication date, and nobody warrants it as one. It is the best signal the API offers; it is not a citable fact. If a specific date matters to you, verify it on the dataset page.

There is no English

One thing that did surprise me: data.gov.il publishes no English metadata at all. Not partial, not patchy — none. I checked all 1,197 packages for a title_translated field, and for anything translation-shaped in extras, and there is nothing there.

So every English title and description in the repo is machine translation, and the Hebrew is the authoritative text. Each month's CSV records which model produced each row, because I don't think you should have to guess: 1,168 came from an earlier snapshot translated with Gemini 2.0 Flash, 23 were done in this repo with GPT-4.1-mini, and six passed through unchanged because the publisher had written a Latin-script title to begin with. Though "not Hebrew" is not the same as "English" — a few of those turn out to be Arabic, Spanish and French.

What it can't see

metadata_created only ever grows, which makes this an additions-only changelog. A dataset that gets withdrawn simply stops appearing in future pulls; it does not generate a "removed in month X" line. Catching deletions needs a diff between two catalogue snapshots (the git history makes that possible; the build doesn't do it yet). Updates to existing datasets are invisible too, and that one is deliberate — metadata_modified churns every time a resource is refreshed and would bury the actual news under noise.

None of which makes it less useful for the thing I wanted it for, which was to stop finding out about public data six months after the fact. The portal will tell you what exists. It just won't tell you when it arrived — and that turned out to be a question worth answering myself.

The repo is at github.com/danielrosehill/Israel-Open-Data-Changelog, rebuildable with a single ./build.sh. The source is data.gov.il, and its API is at /api/3/action/package_search if you'd rather skip my reconstruction and go straight to it.