Green mode

PyTango supports cooperative green Tango objects. Since version 8.1 two green modes have been added: Futures and Gevent. In version 9.2.0 another one has been added: Asyncio.

Note

The preferred mode to use for new projects is Asyncio. Support for this mode will take priority over the others.

The Futures uses the standard python module concurrent.futures. The Gevent mode uses the well known gevent library. The newest, Asyncio mode, uses asyncio - a Python library for asynchronous programming (it’s featured as a part of a standard Python distribution since version 3.5 of Python; it’s available on PyPI for older ones).

You can set the PyTango green mode at a global level. Set the environment variable PYTANGO_GREEN_MODE to either futures, gevent or asyncio (case insensitive). If this environment variable is not defined the PyTango global green mode defaults to Synchronous.

Client green modes

You can also change the active global green mode at any time in your program:

>>> from tango import DeviceProxy, GreenMode
>>> from tango import set_green_mode, get_green_mode

>>> get_green_mode()
tango.GreenMode.Synchronous

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Synchronous

>>> set_green_mode(GreenMode.Futures)
>>> get_green_mode()
tango.GreenMode.Futures

>>> dev.get_green_mode()
tango.GreenMode.Futures

As you can see by the example, the global green mode will affect any previously created DeviceProxy using the default DeviceProxy constructor parameters.

You can specificy green mode on a DeviceProxy at creation time. You can also change the green mode at any time:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Futures

>>> dev.set_green_mode(GreenMode.Synchronous)
>>> dev.get_green_mode()
tango.GreenMode.Synchronous

futures mode

Using concurrent.futures cooperative mode in PyTango is relatively easy:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Futures

>>> print(dev.state())
RUNNING

The tango.futures.DeviceProxy() API is exactly the same as the standard DeviceProxy. The difference is in the semantics of the methods that involve synchronous network calls (constructor included) which may block the execution for a relatively big amount of time. The list of methods that have been modified to accept futures semantics are, on the tango.futures.DeviceProxy():

So how does this work in fact? I see no difference from using the standard DeviceProxy. Well, this is, in fact, one of the goals: be able to use a futures cooperation without changing the API. Behind the scenes the methods mentioned before have been modified to be able to work cooperatively.

All of the above methods have been boosted with two extra keyword arguments wait and timeout which allow to fine tune the behaviour. The wait parameter is by default set to True meaning wait for the request to finish (the default semantics when not using green mode). If wait is set to True, the timeout determines the maximum time to wait for the method to execute. The default is None which means wait forever. If wait is set to False, the timeout is ignored.

If wait is set to True, the result is the same as executing the standard method on a DeviceProxy. If, wait is set to False, the result will be a concurrent.futures.Future. In this case, to get the actual value you will need to do something like:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.state(wait=False)
>>> result
<Future at 0x16cb310 state=pending>

>>> # this will be the blocking code
>>> state = result.result()
>>> print(state)
RUNNING

Here is another example using read_attribute():

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.read_attribute('wave', wait=False)
>>> result
<Future at 0x16cbe50 state=pending>

>>> dev_attr = result.result()
>>> print(dev_attr)
DeviceAttribute[
data_format = tango.AttrDataFormat.SPECTRUM
      dim_x = 256
      dim_y = 0
 has_failed = False
   is_empty = False
       name = 'wave'
    nb_read = 256
 nb_written = 0
    quality = tango.AttrQuality.ATTR_VALID
r_dimension = AttributeDimension(dim_x = 256, dim_y = 0)
       time = TimeVal(tv_nsec = 0, tv_sec = 1383923329, tv_usec = 451821)
       type = tango.CmdArgType.DevDouble
      value = array([ -9.61260664e-01,  -9.65924853e-01,  -9.70294813e-01,
        -9.74369212e-01,  -9.78146810e-01,  -9.81626455e-01,
        -9.84807087e-01,  -9.87687739e-01,  -9.90267531e-01,
        ...
        5.15044507e-1])
    w_dim_x = 0
    w_dim_y = 0
w_dimension = AttributeDimension(dim_x = 0, dim_y = 0)
    w_value = None]

gevent mode

Warning

Before using gevent mode please note that at the time of writing this documentation, tango.gevent requires the latest version 1.0 of gevent (which has been released the day before :-P).

Using gevent cooperative mode in PyTango is relatively easy:

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Gevent

>>> print(dev.state())
RUNNING

The tango.gevent.DeviceProxy() API is exactly the same as the standard DeviceProxy. The difference is in the semantics of the methods that involve synchronous network calls (constructor included) which may block the execution for a relatively big amount of time. The list of methods that have been modified to accept gevent semantics are, on the tango.gevent.DeviceProxy():

So how does this work in fact? I see no difference from using the standard DeviceProxy. Well, this is, in fact, one of the goals: be able to use a gevent cooperation without changing the API. Behind the scenes the methods mentioned before have been modified to be able to work cooperatively with other greenlets.

All of the above methods have been boosted with two extra keyword arguments wait and timeout which allow to fine tune the behaviour. The wait parameter is by default set to True meaning wait for the request to finish (the default semantics when not using green mode). If wait is set to True, the timeout determines the maximum time to wait for the method to execute. The default timeout is None which means wait forever. If wait is set to False, the timeout is ignored.

If wait is set to True, the result is the same as executing the standard method on a DeviceProxy. If, wait is set to False, the result will be a gevent.event.AsyncResult. In this case, to get the actual value you will need to do something like:

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.state(wait=False)
>>> result
<gevent.event.AsyncResult at 0x1a74050>

>>> # this will be the blocking code
>>> state = result.get()
>>> print(state)
RUNNING

Here is another example using read_attribute():

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.read_attribute('wave', wait=False)
>>> result
<gevent.event.AsyncResult at 0x1aff54e>

>>> dev_attr = result.get()
>>> print(dev_attr)
DeviceAttribute[
data_format = tango.AttrDataFormat.SPECTRUM
      dim_x = 256
      dim_y = 0
 has_failed = False
   is_empty = False
       name = 'wave'
    nb_read = 256
 nb_written = 0
    quality = tango.AttrQuality.ATTR_VALID
r_dimension = AttributeDimension(dim_x = 256, dim_y = 0)
       time = TimeVal(tv_nsec = 0, tv_sec = 1383923292, tv_usec = 886720)
       type = tango.CmdArgType.DevDouble
      value = array([ -9.61260664e-01,  -9.65924853e-01,  -9.70294813e-01,
        -9.74369212e-01,  -9.78146810e-01,  -9.81626455e-01,
        -9.84807087e-01,  -9.87687739e-01,  -9.90267531e-01,
        ...
        5.15044507e-1])
    w_dim_x = 0
    w_dim_y = 0
w_dimension = AttributeDimension(dim_x = 0, dim_y = 0)
    w_value = None]

Note

due to the internal workings of gevent, setting the wait flag to True (default) doesn’t prevent other greenlets from running in parallel. This is, in fact, one of the major bonus of working with gevent when compared with concurrent.futures

asyncio mode

Asyncio mode is similar to gevent but it uses explicit coroutines. You can compare gevent and asyncio examples.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import asyncio
from tango.asyncio import DeviceProxy

async def asyncio_example():
    dev = await DeviceProxy("sys/tg_test/1")
    print(dev.get_green_mode())

    print(await dev.state())

loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio_example())
loop.close()

Below you can find a TCP server example, which runs in an asynchronous mode and waits for a device’s attribute name from a TCP client, then asks the device for a value and replies to the TCP client.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
"""A simple TCP server for Tango attributes.

It runs on all interfaces on port 8888:

   $ python tango_tcp_server.py
   Serving on 0.0.0.0 port 8888

It can be accessed using netcat:

   $ ncat localhost 8888
   >>> sys/tg_test/1/ampli
   0.0
   >>> sys/tg_test/1/state
   RUNNING
   >>> sys/tg_test/1/nope
   DevFailed[
   DevError[
        desc = Attribute nope is not supported by device sys/tg_test/1
      origin = AttributeProxy::real_constructor()
      reason = API_UnsupportedAttribute
    severity = ERR]
    ]
   >>> ...
"""

import asyncio
from tango.asyncio import AttributeProxy


async def handle_echo(reader, writer):
    # Write the cursor
    writer.write(b'>>> ')
    # Loop over client request
    async for line in reader:
        request = line.decode().strip()
        # Get attribute value using asyncio green mode
        try:
            proxy = await AttributeProxy(request)
            attr_value = await proxy.read()
            reply = str(attr_value.value)
        # Catch exception if something goes wrong
        except Exception as exc:
            reply = str(exc)
        # Reply to client
        writer.write(reply.encode() + b'\n' + b'>>> ')
    # Close communication
    writer.close()


async def start_serving():
    server = await asyncio.start_server(handle_echo, '0.0.0.0', 8888)
    print('Serving on {} port {}'.format(*server.sockets[0].getsockname()))
    return server


async def stop_serving(server):
    server.close()
    await server.wait_closed()


def main():
    # Start the server
    loop = asyncio.get_event_loop()
    server = loop.run_until_complete(start_serving())
    # Serve requests until Ctrl+C is pressed
    try:
        loop.run_forever()
    except KeyboardInterrupt:
        pass
    # Close the server
    loop.run_until_complete(stop_serving(server))
    loop.close()


if __name__ == '__main__':
    main()

Server green modes

PyTango server API from version 9.2.0 supports two green modes: Gevent and Asyncio. Both can be used in writing new device servers in an asynchronous way.

gevent mode

This mode lets you convert your existing devices to asynchronous devices easily. You just add green_mode = tango.GreenMode.Gevent line to your device class. Consider this example:

class GeventDevice(Device):
    green_mode = tango.GreenMode.Gevent

Every method in your device class will be treated as a coroutine implicitly. This can be beneficial, but also potentially dangerous as it is a lot harder to debug. You should use this green mode with care. Gevent green mode is useful when you don’t want to change too much in your existing code (or you don’t feel comfortable with writing syntax of asynchronous calls).

Another thing to keep in mind is that when using Gevent green mode is that the Tango monitor lock is disabled, so the client requests can be processed concurrently.

Greenlets can also be used to spawn tasks in the background.

asyncio mode

The way asyncio green mode on the server side works is it redirects all user code to an event loop. This means that all user methods become coroutines, so in Python > 3.5 you should define them with async keyword. In Python < 3.5, you should use a @coroutine decorator. This also means that in order to convert existing code of your devices to Asyncio green mode you will have to introduce at least those changes. But, of course, to truly benefit from this green mode (and asynchronous approach in general), you should introduce more far-fetched changes!

The main benefit of asynchronous programing approach is that it lets you control precisely when code is run sequentially without interruptions and when control can be given back to the event loop. It’s especially useful if you want to perform some long operations and don’t want to prevent clients from accessing other parts of your device (attributes, in particular). This means that in Asyncio green mode there is no monitor lock!

The example below shows how asyncio can be used to write an asynchronous Tango device:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
"""Demo Tango Device Server using asyncio green mode"""

import asyncio
from tango import DevState, GreenMode
from tango.server import Device, command, attribute


class AsyncioDevice(Device):
    green_mode = GreenMode.Asyncio

    async def init_device(self):
        await super().init_device()
        self.set_state(DevState.ON)

    @command
    async def long_running_command(self):
        self.set_state(DevState.OPEN)
        await asyncio.sleep(2)
        self.set_state(DevState.CLOSE)

    @command
    async def background_task_command(self):
        loop = asyncio.get_event_loop()
        future = loop.create_task(self.coroutine_target())

    async def coroutine_target(self):
        self.set_state(DevState.INSERT)
        await asyncio.sleep(15)
        self.set_state(DevState.EXTRACT)

    @attribute
    async def test_attribute(self):
        await asyncio.sleep(2)
        return 42


if __name__ == '__main__':
    AsyncioDevice.run_server()