Systems only become truly valuable when they are connected. This way they can incorporate input from other systems, help automate processes, and you can analyse their data.

In a previous blog post, we showed you how to take this into account when you want to ensure that a system you purchase is properly accessible. But if you build systems yourself that you want to set up for this purpose, it also raises questions: how do these work and how do you make this process as seamless as possible? I would like to explain more about this in this blog post.

How does an API work?

The way we connect systems is via APIs, or Application Programming Interfaces. These consist of a specification, essentially an agreement between the producer and consumer of data. Such an agreement can then be implemented from both sides. You also speak of API servers and API clients.

In the popular REST standard, services can communicate over the internet via Hypertext Transfer Protocol (HTTP), where they address each other with a request consisting of a URL (or more broadly: URI) together with headers, a verb (such as GET, POST) and depending on that, further data, subsequently answered with a response also furnished with headers together with a machine-friendly status code, such as 200 (OK) or 404 (Not Found). REST requests are not remembered via sessions, so each request must authenticate anew, usually via the Open Authorization (OAuth) standard.

For data exchange, there are various formats. Since the rise of the modern web browser, the most popular option has become JSON, or JavaScript Object Notation. Where this is more robustly specified than the spreadsheet format Comma-Separated Values (CSV), machine-readable validation schemas are not baked in, whereas this was the norm with XML-based SOAP, which was popular in the Java world.

Describing APIs in schemas

Specifying formats in JSON is nevertheless possible via JSON Schema, where you can indicate what object keys you use and what data types lie behind them. API responses are described with OpenAPI (formerly Swagger), allowing you to describe what answers you may expect (with which status code) at each API endpoint, consisting of an HTTP verb at a URL path and how requests to it are structured.

The handy thing about this is that, because it is machine-readable, much has been built around it. You can have responses checked against this agreed format, generate interactive documentation where a user can even immediately try out requests to your API, but also create sample data or (web) forms that deliver data in your format, or even generate code for database schemas, for such API clients, servers - or the reverse.

Not everything in these specifications is standardised yet, and you need to make clear yourself, for example, how frequently your API may be called (rate limits) or how requests split a larger result into separate pages. Such pagination is often handled via an 'envelope', or a standard wrapper for your data with a place for this kind of metadata. Rate limits can be communicated via HTTP headers, a robots.txt file, or documentation.

You also retain choice over the URL structure, although it is common to handle filters and pagination via query parameters, such as /invoices/?paid=true&limit=50&offset=100 for paid invoices from the 101st to the 150th (within your default sorting method).

ID numbers that refer to a fixed object can go in URL path parameters such as /clients/123/invoices/456 for invoice 456 of client 123. This is also immediately handy for your Search Engine Optimisation (SEO), because search engines then know that the latter is probably worth indexing individually, whereas the former probably is not.

Conclusion and tips

Despite limitations, it is therefore useful to describe your API in an OpenAPI schema, because this makes much of the peripheral work easier for both the producer and the consumer of the data. It can easily happen that details are forgotten, for example that a field can also have missing data (in JSON: null) or to update such a schema when a field is added. For this, it can be useful to apply one of the existing implementations around data validation via OpenAPI or JSON Schema, so that you can automatically check whether what you receive (and what you return) still matches the described agreements.

Over time, it is naturally also to be expected that such agreements evolve. It is then handy to keep changes compatible with previous versions where possible, or for major changes to place new versions at a new URL, such as /v2/invoices/. For consumers who want to stay informed about how, when, and why things change exactly, it is finally nice to have a place where they can track such updates. You can maintain your OpenAPI schema via version control systems like Git and publish on websites such as GitHub or Codeberg, where developers can choose to follow such updates and compare differences.

Want more information about data-driven work? Keep following our blog for new posts!