This month's data snapshot highlights information regarding the development and implementation of the new Iowa Data Hub, including: identifying authorizers, using record IDs, and supporting source platforms and file types.
Identifying Authorizers
In last month's snapshot, we outlined the proposed roles for the new Iowa Data Hub. Over the next few weeks DOM will be reaching out to agencies to name individuals to serve as authorizers for each organization in the proposed organizational hierarchy (available under related files). Two of the key responsibilities of authorizers are:
- Authorizing content owned by their organization (e.g., datasets, views, looks & dashboards) to be shared with other state organizations or publicly.
- Approving user requests for role assignment in their organization.
Authorizers are critical in governing access to their organization's published data assets.
Requiring Record IDs
In the New Iowa Data Hub, datasets will be required to have record IDs. The record IDs serve as a unique identifier for each record in the dataset. The primary purpose of using record IDs will be to improve data integrity by preventing duplicate records. They will also facilitate processing updates.
DOM will be reaching out to publishers of datasets to identify options for adding or constructing record IDs for datasets currently published within our current Iowa Enterprise Data application.
Source Platforms and File Types
The new Iowa Data Hub is planning to support a variety of source platforms and source file types for creating and updating published datasets.
MSSQL & MYSQL Databases
We are developing the ability to set up data connections to tables and/or views within Microsoft SQL and MySQL databases. Publishers will have the ability to update datasets on demand or on a defined schedule. Support for full replaces and upsert (new or changed records) is planned.
Google Drive/One Drive
The plan is to support connecting to the following file types stored in either Google Drive or One Drive: CSV, Excel (XLSX), Google Sheet, JSON, GEOJSON, and Shapefile. Publishers will be able to update datasets on demand or on a defined schedule. Full replaces will be required for all updates from Google Drive and One Drive.
Network/Local Drive
Publishers will be able to create and update datasets utilizing the following file types: CSV, Excel (XLSX), JSON, GEOJSON, and Shapefile. Updates from a network/local drive must be on demand. CSV files will support both full replace and upsert (new or changed records). All other file types will support full replace.
ArcGIS REST Services Directory
Publishers will be able to connect to layers made available in a public ArcGIS Rest Services Directory. Publishers will be able to update on demand or based on a defined schedule. Only full replaces will be supported.
Google Cloud Storage
Google Cloud Storage will be used to support updates based on automated file transfers of the file types noted with the network/local drive. Publishers will be able to update datasets on demand or on a defined schedule. CSV files will support both full replace and upsert (new or changed records). All other file types will support full replace.