How to import multiple device classes to the Catalogue
Intended audience: administrators
The Device Classes Catalogue is available on the Tango Controls web page: http://www.tango-controls.org/.
To import multiple device classes to the catalogue from a repository please use an import script.
Requirements
Python2.7
Subversion
SVN python library: pip install svn
Warning
The library will not work if any of files in the SVN has no author defined which is the case for tango-ds repository on the Sourceforge. To avoid problems one can edit
svn/common.py
around line 358 to have something like the following:author = '' if commit_node.find('author') is not None: author = commit_node.find('author').text
urllib2 library if it is not installed with Python itself: pip install urllib2
Requests library version >= 2.12, you may need to run pip with –upgrade option: pip install --upgrade Requests
How-to import multiple classes
Install packages listed in Requirements if not yet installed
- Create a folder for a local copy of a repository:
example: mkdir ~/tmp/local-repo
- Clone the import utility with git:
git clone https://github.com/piogor/dsc-import.git
- Get into cloned sources:
cd dsc-import
- Make your local branch to be sure your settings will not be overwritten by someone else.
git checkout -b my_local_branch
Update variables in a file
dsc_import_utility.py
to reflect your environment:FORCE_UPDATE = False # when True no timestamps are checked and updates are performed TEST_SERVER_AUTH = False # Set true if script is run against test server with additional authentication (webu test) VERIFY_CERT = False # set this to false if running aginst test server without a valid certificate USE_DOC_FOR_NON_XMI = True # when True, parse documentation to get xmi conntent for device servers without XMI ADD_LINK_TO_DOCUMENTATION = True # when True it provides a link to documentation # set the following variables to point to the repositories LOCAL_REPO_PATH = '/home/piotr/tmp/tango-ds-repo/' # local copy of the repository will be synced there LOG_PATH = '/home/piotr/tmp' # where to log some information about import process, not used now. REMOTE_REPO_HOST = 'svn.code.sf.net' # host of the SVN repository REMOTE_REPO_PATH = 'p/tango-ds/code' # path within the server where the repository is located # if one would like to limit a search tree (useful for one device server update and/or tests) REPO_START_PATH = 'DeviceClasses' # do not provide start nor end slashes # Tango Controls or test server address SERVER_BASE_URL = 'http://www.tango-controls.org/'
run with a command: python dsc_import_utility.py
Note
It will ask you for your credentials for tango-controls.org and import/update device classes using provided account.
How the script works
It does import in the following way:
It makes a local copy (in path defined by LOCAL_REPO_PATH) of a SVN repository to speed up search procedure.
Then it searches the local copy for folders containing .XMI files. It takes into account the standard branches/tags/trunk structure. The folders where it findes .xmi files or a proper structure are listed as candidates to be device servers.
Then, the list of candidates then is processed and compared (by repository URL) with content in the Device Classes Catalogue.
If there are changes or FORCE_UPDATE is True the catalogue is updated
For device server without .XMI file it looks for documentation server and tries to parse html documentation generated by Pogo.
If there ara no changes the device server is skipped
These information you will find in a README.rst
( https://github.com/piogor/dsc-import/blob/master/README.rst ).