Data Transfer Project – Break the “data black hole” of information silos

Many online service providers provide us with various online tools such as netbook, email, instant messaging, etc. However, due to privacy and security and competition, most of the data of online tools (such as email, photos, address book, etc.) cannot be shared with each other. Earlier, Google, Microsoft and other technology giants jointly released the “Data Transfer Project” (Data Transfer Project), which allows the transfer of data between different network platforms without barriers, let’s see what the mystery behind it.

Information silos, how network service providers can protect their own data

As mentioned above, most of the data of network tools cannot be interconnected with each other due to privacy and security, competition and other factors. Because each company uses different technologies and development and encryption protocols for their own network software, these protocols are not compatible with each other, so most of them cannot interoperate with each other.

For example, Google and Microsoft both provide calendar functions, but by default we cannot import Microsoft calendar functions in Google Gmail. In order to facilitate other web tools to access their data, service providers generally share data through open APIs (Application Programming Interface), which is a piece of written code or compiled program that is made available to third parties so that they can use the API to The API (Application Programming Interface) is a piece of code written or compiled to be used by a third party so that the third-party program can access specific data using the API (Figure 1).


Figure 1 Illustration of API

API allows third parties to call data of their own developed software, but traditional API can only realize one-way call, for example, company B can access the data of user C through company A’s API, but company A cannot access the data of company B. If you want to realize A’s access to B, B must develop the corresponding API. For example, company B can access user C’s data through company A’s API, but company A cannot access company B’s data, so if we want to realize A’s access to B, B must develop the corresponding API again, so each company needs to develop API, and obviously it is extremely inconvenient to share data with each other. For this reason, Google, Microsoft, and others have proposed the Data Transfer Project, which can realize bidirectional and multidirectional data sharing among major companies (Figure 2).


Figure 2 Data Transfer Project plan

Behind the sharing, the principle of Data Transfer Project is explained

As mentioned above, Google and Microsoft have proposed the Data Transfer Project in order to facilitate data sharing among major network software. How does the Data Transfer Project share data with each other?

The first step of this project is to establish a data model, the role of this model is to develop a common standard format for data transfer and reading of different service providers. The main tool used here is an adapter that converts each user’s private data (e.g., email, calendar) and authentication format (e.g., account number, password) into a uniform form that can be used by all service providers. User data models are aggregated together, mostly by industry grouping, to form vertical data. Users can have data in one or more verticals, for example a vertical search could be photos, email, contacts or music. This allows the data model to turn the user’s data across different service providers into common data that is readable to each other (Figure 3).


Figure 3 Data Model

Data Transfer Project then uses “data adapters” and “identity adapters” to read and authenticate data. The Data Adapter converts each company’s original API into a code snippet of the data model that Data Transfer Project can use, so that the data stored in it can be read by users on web tools developed by other service providers. The authentication adapter allows users to authenticate their accounts before transferring data out of one service provider or to another (Figure 4).


Figure 4 Adapter Role

The last thing left is the transfer pipeline. Data Transfer Project has developed a set of task management libraries to manage the transfer of data, while using the same encryption protocol to secure the data transfer during the transfer process.

In this way, Data Transfer Project leverages a common framework (data model) and ecosystem (data adapters and authentication adapters) to complete the authentication of the user, and then the user’s private data can be seamlessly migrated from one service provider to another (Figure 5).


Figure 5 Data Transfer Project plans to enable two-way data migration

Data sharing to facilitate our life

Nowadays, almost every Internet user has many Internet accounts, and we will save many private files in each account, such as incoming and outgoing emails, photos taken, programs downloaded, and other resources. However, if we want to stop a certain internet service before, for example, we don’t want to use Gmail service and want to use Microsoft service instead, then we need to download the emails, calendars, photos, etc. saved in our Google account and then upload them to our Microsoft account in order to To complete the account migration work.

Now with Data Transfer Project service, we just need to initiate a request on our Google account and use the migration service provided by Data Transfer Project to migrate all these data to Microsoft servers from our Microsoft account. This data migration is much more convenient. Of course, as a multi-party sharing data service, Data Transfer Project will provide more services in the future, such as laptop facial recognition files, fingerprint payment profiles and other data files can be migrated, which means that we just need to initially set up a file, and later replace the hardware or network account can continue to use, greatly facilitating our network life. .

Leave a Comment