How do I import a new dataset on the Open Data Portal?
Agency data coordinators and data stewards are able to create new datasets on data.iowa.gov through a web-based interface. To import datasets via the web-based interface, you will first need to be logged into the open data portal (data.iowa.gov). Once signed in, head to your dashboard (you should be automatically directed there once you sign in). If you are not on your dashboard, you can easily navigate to your dashboard by clicking the “Hello” link above the main search box for the website. Once you are on your dashboard, click the “Create a New Dataset” button to get started, then complete the following:
- Select import option
- Review data file schema
- Create location column (if applicable)
- Provide metadata and set permissions
- Change column properties
- Publish dataset
Agency data coordinators and data stewards will have a few options available to them when creating a new dataset, as shown in the following image. Below are the descriptions for each:
- “Design from Scratch” allows you to create the dataset schema without importing the data. This should be used where the data files are more than 250,000 rows or 150 MB, and/or are going to be frequently updated, and will imported and updated automatically, see Automating Updates.
- “Import a Data File” allows you to import data (where file is less than 250,000 rows or 150 MB) and create the dataset schema. Most agencies will use this option. When you select this option you will be prompted to identify the location of the dataset – whether it is on your computer or on the internet.
- “Connect with an ESRI Map Layer” are for those agencies with Map Server Layer.
- “Import Geospatial Data” is used to import a Shapefile or Keyhole Markup Language file.
- “Upload a Non-Data File” and “Link to External Data” will generally not be used. However, there may be instances where it makes the most sense. Most notably would be linking to external data to catalog public online databases. Agencies should consult with the state data administrator if you believe this choice would be the best option for you.
The system makes an educated guess on the data type based data contained in each column. However, you are able to change it if it is not correct. The system supports variety of data types. Although it is possible to change the data types later on, it can be difficult if the dataset is large (e.g. > 50,000 rows) and can result in loss of data. It is best to review and correct when setting up the dataset originally. This step is only for those who "Import a Data File".
When importing your data, you will need to create a location column with any geospatial data you may have within the CSV file, such as address columns and latitudes and longitudes. Once a location column has been added you will have the ability to point to either a single column or multiple columns for which coordinates will be derived. In most cases, you will have multiple columns, and can specify the columns that contain the latitude and longitude, or street address, city, state and zip. This step is only for those who "Import a Data File".
Once you review the data schema and create any applicable locations columns, you will be presented a web form contains metadata fields. However, not all metadata is available to you on the initial import. You are encouraged to edit your metadata later on to provide all the necessary information to ensure the public understands what your data is all about.
On this form, you also have the opportunity to indicate if your dataset is public or private. Data published to data.iowa.gov should be marked as “public” when importing your data. It is important to note that even though it is “public” your new dataset will not be accessible to the public until it has been published.
To change column properties, simply move your mouse arrow to the column you wish the change. On the right side of the column, you will see a menu icon, click on it, see figure 6. A drop box will appear, select the “Edit Column Properties” option, see image to right. This will open the Column Properties panel where you can rename the column (i.e. provide a label for the column), provide a description for the column, and change the column’s formatting. See data dictionary for more details and guidance on what to include.
If you "Connect with a ESRI Map Layer", you will not be able to edit column properties.
New datasets are working copies that are not accessible to the public until you publish it. This will allow you to share datasets for review and collaboration until it is ready to be made public. Before you can make your dataset public, you'll need to publish your working copy to make a published copy of the dataset for the public to access. To publish your dataset, just click “Publish Dataset” button adjacent to the dataset title.
|Program Area|| |
Open Data, Importing Data, New Dataset