It was also a side project to start tinkering with next. I found it had a nice DX and allowed me to quickly build it.
Now I think it might also be possible to just use something like Astro as I don't need many dynamic SSR features.
> Now I think it might also be possible to just use something like Astro as I don't need many dynamic SSR features.
You're doing a very useful service to the community, which is absolutely praise-worthy; please don't listen too much to my moaning about next.js. As long as it does the job, you're gold.
I built this tool over the years primarily for myself.
I had some questions about political finance flows that were impossible to answer just by staring at the raw tables published by governments.
I needed a way to visualize the insights for my own curiosity. Since the result was useful, I polished it up and published it so everyone else can explore the data too.
The Architecture (No Database):
To keep it fast and low-maintenance, there is no running database.
- backend: The scrapers are run locally and pre-generate the entire dataset into static JSON files
- frontend: The client loads these JSON chunks on demand. Many pages are staticly pre-rendered on build.
- stack: It’s a next.js app, hosted via OpenNext on Cloudflare Workers.
Moving Next.js to the edge (using OpenNext) combined with a purely static data model means the site is cheap to host and scales instantly, even when a big new dataset drops.
The "Open Data" reality:
While building the scrapers to answer my questions, I encountered some data quirks. A few favorites:
- The Greek Belgian: One 2021 EU foundation document randomly used Greek unicode characters (ΒΕ) instead of ASCII (BE) for the Belgium country code.
- The Encoding: All Austrian CSVs were standard UTF-8, except the 2022 file which was randomly encoded in Mac OS Roman (Apple Macintosh).
- The Vanishing Files: The Croatian Electoral Commission’s database has 404 errors for specific 2019/2020 party donation lists that used to be there. They made them available again after being informed about this issue.
I report these quirks upstream so the next developer doesn't have to suffer. I keep a log of the weirdest ones here: https://donation.watch/en/fun.
I’ve finally cleaned up the code a bit and open-sourced the whole thing (AGPL-3.0 for code, CC BY 4.0 for the data).
I'm looking to add more countries. Do you know of any additional countries public donation datasets that are actually machine-readable (CSV/JSON/API)?
I've already investigated a few that are currently too messy (e.g., scanned PDFs or bad formats), listed here: https://donation.watch/en/other-countries.
I'd like to integrate the US dataset in the future. The issue was that it's really big and I still have to figure out a good way to filter uninteresting donations out.
We [1] use ECharts to visualize all of our data and it's pretty great to use.
It's also very pluggable, so one can only import the components one needs for each chart.
Meaning if you only use the `bar` series type, you can just only import the `bar` component. Same thing not only for other chart types but also stuff like mark areas or zoom controls.
There's also a bigger update in the work which brings e.g. violin plots.
I'll write them to ask if they're ok with me using their data.
I've previously found https://finantarepartide.ro/partidul-pro-romania-pro-romania... which should be the official romanian source. When attempting to integrate them I found some missing links tho and they never replied to my mail :D
Regarding next:
It was also a side project to start tinkering with next. I found it had a nice DX and allowed me to quickly build it. Now I think it might also be possible to just use something like Astro as I don't need many dynamic SSR features.