Many people have asked for an automated way to update the data contained in their Good Data projects. Attached at the bottom of this post is a Python script that allows just that. 

This script allows you to upload data to existing datasets. You need to first create the datasets by uploading an initial CSV files (perhaps with dummy data). The script supports both incremental and full data loads. Feel free to contact us on support@gooddata.com if you have any questions regarding the script.

Note: this script is a pre-release version that is not fully supported. We have done basic testing and everything appears to work fine, but as of yet, this is an unsupported product. If you have problems with this script, we recommend using the standard UI for your uploads.

 Release Date

 Updated on August 25, 2010

Release Notes

2010-03-09
-n switch added for testing upload only and finish before integration.

2009-11-27

fixed problem related to client-side timing out after long downloads

2009-09-09

minor changes added to make script compatible with the newest release of the Good Data application

2009-08-13

-i switch added for incremental load, default is full load

2009-04-24

simplejson library is needed on python2.5 - http://www.undefined.org/python/, http://pypi.python.org/pypi/simplejson

2009-04-24

The REST protocol is as follows:

--> POST /gdc/account/login {postUserLogin: {login: '...', password: '...', remember: 0/1}}

<-- GDCAuthSST (cookie)

--> GET /gdc/account/token

<-- GDCAuthTT (cookie) (both these cookies have to be used for all subsequent requests to the server)

--> POST /gdc/md/<project>/etl/uploadfile multipart/form-data (name of the chunk with CSV file must be uploadFile)

<-- (structure describing the etlFile)

--> GET /gdc/md/<project>/query/etlinterfaces (find the appropriate interface by title)

--> POST /gdc/md/<project>/etl/integrationfile {postIntegrationFile: {etlfile: '...', etlinterface: '...'}} (URIs of etlfile and etlinterface)

Periodically check:

--> GET /gdc/md/<project>/query/etlinterfaces (look for appropriate etlinterface

    and item in its uploads array to get progress and status)

2009-04-21

This is prototype of reference Python implementation of utils using our REST

API.

To get the script, right click the file link below and select "Save As..."