The election of Trump has had major consequences worldwide, not least for organisations that process (sensitive) data. How this affects us and what we can do…

Clients increasingly ask us: how do we keep our data safe, away from (for example) the United States government? Simultaneously, we receive the question: how can we migrate to the cloud as quickly as possible, particularly Microsoft Azure? Our answer? Look at the European cloud - and always be prepared for a swift move, should it prove necessary.

What is the real risk of data in the cloud?

When it comes to data warehousing, we hear statements like these all too often:

  • It is safer, because Microsoft/Google/AWS certainly know what they are doing
  • It is cheaper, because you can benefit from economies of scale
  • The data centre is in the Netherlands, so the data falls under Dutch law
  • It is safe, because we tick 'encryption at rest'

Unfortunately, all these statements are incorrect. Of course, large cloud providers know exactly what they are doing and data leaks are exceptionally bad for reputation. Therefore, much is invested in the security of data storage and processing in the cloud. This applies equally to the large European cloud providers such as OVHcloud (France) and StackIT (Germany). But it is a bad idea to outsource the security of your cloud entirely to your cloud provider: one wrongly placed checkbox and all your data is publicly available worldwide.

And regarding costs: cheaper it certainly is not, unless you know precisely how to optimise your usage. And for that, substantive expertise is an important requirement - contrary to the promise that the cloud would make everything simpler.

Many cloud providers have data centres in the Netherlands or elsewhere in the European Union. That seems safe, as the data is on European soil. For two reasons, this is unfortunately a fairy tale: data continuously flies around the world and is therefore not limited to the data centre where it is stored. Moreover, the US government states that all American companies fall under US legislation, so they must - on request - hand over data. Because the interests of cloud providers with the US government are virtually always greater than those of European businesses or governments, such requests are virtually always complied with.

The last statement - it is safe because encryption is switched on - is perhaps the most dangerous. In virtually every cloud environment, you can secure your data through encryption. It is important to realise that a distinction exists between encryption-in-transit (when data travels over the internet, it is secured so no one can read it) and encryption-at-rest (data stored in the cloud is secured with encryption). The first form of security is a no-brainer and truly helps. But the second form is largely a fig leaf: data secured with encryption indeed cannot be read by others. But to work with the data, the encryption keys must virtually always be stored with the cloud provider itself. It is somewhat like having heavy metal doors installed in your house, but immediately giving them the key. Conclusion? Encryption virtually never fully protects you against governments or software providers who want access to your data.

Thanks to Trump, finally attention for 'vendor lock-in'

Storing data with American providers is clearly not such a good idea. Are European cloud providers a good alternative? That depends somewhat. For many basic matters such as compute (server power), databases (storage of structured data) and data lakes (storage of unstructured data), European providers can excellently compete with their American counterparts. For specific applications it can be harder - for instance, Azure Data Factory (an application for unlocking data from source systems into a data warehouse) is - as the name suggests - only available from Microsoft. This now forces you to think about vendor lock-in.

Besides data security, another issue plays a role: once you have housed your infrastructure with one provider, it is virtually impossible to switch to another. Azure Data Factory works slightly differently from Amazon's Glue. Redshift and Google BigQuery work differently again from Microsoft Fabric. In short: vendor lock-in!

Cloud providers know this. They lure you with attractive starting rates because they know: once inside, never leaving. Companies like Oracle and SAP are known for this practice.

Two solutions: portability and the European cloud

The most important solution against both vendor lock-in and data security lies in the portability of the data platform we use. A data platform is portable if it meets the following requirements:

  • The platform does not use systems that are only available from one provider.
  • It is easy to move the platform between various cloud platforms.
  • It is possible to run the platform on-premises as well.

Are you about to invest in a cloud platform and do you find data security and preventing vendor lock-in important? Then pay particular attention to the points above. Unsure? Feel free to engage us for independent advice.

Technical deep-dive: how do you arrange portability technically?

At Datalab, we have developed a 100% portable data platform: Datalab Studio. From the first development, we set the hard requirement: it must run in Azure, but also with Amazon, Google and on-premises. We achieved this through a number of important conditions:

  • All applications run in so-called containers. Because this is a worldwide standard, we can run the containers in virtually all cloud and on-premises environments.
  • We have a clear preference for open-source solutions, such as Apache NiFi (as an alternative to Azure Data Factory and Amazon Glue), Airflow (as an alternative to Databricks), PostgreSQL (as an alternative to Microsoft SQL Server), Metabase (as an alternative to Power BI and Tableau), and numerous other systems.

Besides portability, the choice of cloud provider plays an important role. We increasingly advise our clients to use a European provider, such as the aforementioned OVHcloud or StackIT. If this is not an option, the advice is: ensure you are prepared for a swift relocation of your platform, for example by applying the techniques mentioned above.

Sitting back and hoping everything will be fine is no longer an option. Under GDPR, you are obliged to handle data carefully. Just as you would not leave the doors of your server room unattended and open, you should not leave the doors to your data open either.

Numerous organisations, from governments to medical institutions, that work with highly sensitive data already make use of Datalab's advisory and/or implementation role. Want to know whether we can also offer a good alternative for your data-technical challenges? Have a no-obligation conversation with us!