How to import multiple device classes to the Catalogue

Intended audience: administrators

The Device Classes Catalogue is available on the Tango Controls web page: http://www.tango-controls.org/.

To import multiple device classes to the catalogue from a repository please use an import script.

Requirements

  • Python2.7

  • Subversion

  • SVN python library: pip install svn

    Warning

    The library will not work if any of files in the SVN has no author defined which is the case for tango-ds repository on the Sourceforge. To avoid problems one can edit svn/common.py around line 358 to have something like the following:

    author = ''
if commit_node.find('author') is not None:
    author = commit_node.find('author').text

  • urllib2 library if it is not installed with Python itself: pip install urllib2

  • Requests library version >= 2.12, you may need to run pip with –upgrade option: pip install --upgrade Requests

How-to import multiple classes

  1. Install packages listed in Requirements if not yet installed

  2. Create a folder for a local copy of a repository:
    • example: mkdir ~/tmp/local-repo
  3. Clone the import utility with git:
    • git clone https://github.com/piogor/dsc-import.git
  4. Get into cloned sources:
    • cd dsc-import
  5. Make your local branch to be sure your settings will not be overwritten by someone else.
    • git checkout -b my_local_branch

  6. Update variables in a file dsc_import_utility.py to reflect your environment:

    FORCE_UPDATE = False  # when True no timestamps are checked and updates are performed
TEST_SERVER_AUTH = False  # Set true if script is run against test server with additional authentication (webu test)
VERIFY_CERT = False  # set this to false if running aginst test server without a valid certificate
USE_DOC_FOR_NON_XMI = True # when True, parse documentation to get xmi conntent for device servers without XMI
ADD_LINK_TO_DOCUMENTATION = True # when True it provides a link to documentation

# set the following variables to point to the repositories
LOCAL_REPO_PATH = '/home/piotr/tmp/tango-ds-repo/'  # local copy of the repository will be synced there
LOG_PATH = '/home/piotr/tmp'  # where to log some information about import process, not used now.

REMOTE_REPO_HOST = 'svn.code.sf.net'  # host of the SVN repository
REMOTE_REPO_PATH = 'p/tango-ds/code'  # path within the server where the repository is located

# if one would like to limit a search tree (useful for one device server update and/or tests)
REPO_START_PATH = 'DeviceClasses'  # do not provide start nor end slashes

# Tango Controls or test server address
SERVER_BASE_URL = 'http://www.tango-controls.org/'

  7. run with a command: python dsc_import_utility.py

    Note

    It will ask you for your credentials for tango-controls.org and import/update device classes using provided account.

How the script works

It does import in the following way:

  • It makes a local copy (in path defined by LOCAL_REPO_PATH) of a SVN repository to speed up search procedure.
  • Then it searches the local copy for folders containing .XMI files. It takes into account the standard branches/tags/trunk structure. The folders where it findes .xmi files or a proper structure are listed as candidates to be device servers.
  • Then, the list of candidates then is processed and compared (by repository URL) with content in the Device Classes Catalogue.
    • If there are changes or FORCE_UPDATE is True the catalogue is updated
      • For device server without .XMI file it looks for documentation server and tries to parse html documentation generated by Pogo.
    • If there ara no changes the device server is skipped

These information you will find in a README.rst ( https://github.com/piogor/dsc-import/blob/master/README.rst ).