Green mode

PyTango supports cooperative green Tango objects. Since version 8.1 two green modes have been added: Futures and Gevent. In version 9.2.0 another one has been added: Asyncio.

Note

The preferred mode to use for new projects is Asyncio. Support for this mode will take priority over the others.

The Futures uses the standard python module concurrent.futures. The Gevent mode uses the well known gevent library. The newest, Asyncio mode, uses asyncio - a Python library for asynchronous programming (it’s featured as a part of a standard Python distribution since version 3.5 of Python; it’s available on PyPI for older ones).

You can set the PyTango green mode at a global level. Set the environment variable PYTANGO_GREEN_MODE to either futures, gevent or asyncio (case insensitive). If this environment variable is not defined the PyTango global green mode defaults to Synchronous.

Client green modes

You can also change the active global green mode at any time in your program:

>>> from tango import DeviceProxy, GreenMode
>>> from tango import set_green_mode, get_green_mode

>>> get_green_mode()
tango.GreenMode.Synchronous

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Synchronous

>>> set_green_mode(GreenMode.Futures)
>>> get_green_mode()
tango.GreenMode.Futures

>>> dev.get_green_mode()
tango.GreenMode.Futures

As you can see by the example, the global green mode will affect any previously created DeviceProxy using the default DeviceProxy constructor parameters.

You can specificy green mode on a DeviceProxy at creation time. You can also change the green mode at any time:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Futures

>>> dev.set_green_mode(GreenMode.Synchronous)
>>> dev.get_green_mode()
tango.GreenMode.Synchronous

futures mode

Using concurrent.futures cooperative mode in PyTango is relatively easy:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Futures

>>> print(dev.state())
RUNNING

The tango.futures.DeviceProxy() API is exactly the same as the standard DeviceProxy. The difference is in the semantics of the methods that involve synchronous network calls (constructor included) which may block the execution for a relatively big amount of time. The list of methods that have been modified to accept futures semantics are, on the tango.futures.DeviceProxy():

So how does this work in fact? I see no difference from using the standard DeviceProxy. Well, this is, in fact, one of the goals: be able to use a futures cooperation without changing the API. Behind the scenes the methods mentioned before have been modified to be able to work cooperatively.

All of the above methods have been boosted with two extra keyword arguments wait and timeout which allow to fine tune the behaviour. The wait parameter is by default set to True meaning wait for the request to finish (the default semantics when not using green mode). If wait is set to True, the timeout determines the maximum time to wait for the method to execute. The default is None which means wait forever. If wait is set to False, the timeout is ignored.

If wait is set to True, the result is the same as executing the standard method on a DeviceProxy. If, wait is set to False, the result will be a concurrent.futures.Future. In this case, to get the actual value you will need to do something like:

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.state(wait=False)
>>> result
<Future at 0x16cb310 state=pending>

>>> # this will be the blocking code
>>> state = result.result()
>>> print(state)
RUNNING

Here is another example using read_attribute():

>>> from tango.futures import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.read_attribute('wave', wait=False)
>>> result
<Future at 0x16cbe50 state=pending>

>>> dev_attr = result.result()
>>> print(dev_attr)
DeviceAttribute[
data_format = tango.AttrDataFormat.SPECTRUM
      dim_x = 256
      dim_y = 0
 has_failed = False
   is_empty = False
       name = 'wave'
    nb_read = 256
 nb_written = 0
    quality = tango.AttrQuality.ATTR_VALID
r_dimension = AttributeDimension(dim_x = 256, dim_y = 0)
       time = TimeVal(tv_nsec = 0, tv_sec = 1383923329, tv_usec = 451821)
       type = tango.CmdArgType.DevDouble
      value = array([ -9.61260664e-01,  -9.65924853e-01,  -9.70294813e-01,
        -9.74369212e-01,  -9.78146810e-01,  -9.81626455e-01,
        -9.84807087e-01,  -9.87687739e-01,  -9.90267531e-01,
        ...
        5.15044507e-1])
    w_dim_x = 0
    w_dim_y = 0
w_dimension = AttributeDimension(dim_x = 0, dim_y = 0)
    w_value = None]

gevent mode

Warning

Before using gevent mode please note that at the time of writing this documentation, tango.gevent requires the latest version 1.0 of gevent (which has been released the day before :-P).

Using gevent cooperative mode in PyTango is relatively easy:

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> dev.get_green_mode()
tango.GreenMode.Gevent

>>> print(dev.state())
RUNNING

The tango.gevent.DeviceProxy() API is exactly the same as the standard DeviceProxy. The difference is in the semantics of the methods that involve synchronous network calls (constructor included) which may block the execution for a relatively big amount of time. The list of methods that have been modified to accept gevent semantics are, on the tango.gevent.DeviceProxy():

So how does this work in fact? I see no difference from using the standard DeviceProxy. Well, this is, in fact, one of the goals: be able to use a gevent cooperation without changing the API. Behind the scenes the methods mentioned before have been modified to be able to work cooperatively with other greenlets.

All of the above methods have been boosted with two extra keyword arguments wait and timeout which allow to fine tune the behaviour. The wait parameter is by default set to True meaning wait for the request to finish (the default semantics when not using green mode). If wait is set to True, the timeout determines the maximum time to wait for the method to execute. The default timeout is None which means wait forever. If wait is set to False, the timeout is ignored.

If wait is set to True, the result is the same as executing the standard method on a DeviceProxy. If, wait is set to False, the result will be a gevent.event.AsyncResult. In this case, to get the actual value you will need to do something like:

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.state(wait=False)
>>> result
<gevent.event.AsyncResult at 0x1a74050>

>>> # this will be the blocking code
>>> state = result.get()
>>> print(state)
RUNNING

Here is another example using read_attribute():

>>> from tango.gevent import DeviceProxy

>>> dev = DeviceProxy("sys/tg_test/1")
>>> result = dev.read_attribute('wave', wait=False)
>>> result
<gevent.event.AsyncResult at 0x1aff54e>

>>> dev_attr = result.get()
>>> print(dev_attr)
DeviceAttribute[
data_format = tango.AttrDataFormat.SPECTRUM
      dim_x = 256
      dim_y = 0
 has_failed = False
   is_empty = False
       name = 'wave'
    nb_read = 256
 nb_written = 0
    quality = tango.AttrQuality.ATTR_VALID
r_dimension = AttributeDimension(dim_x = 256, dim_y = 0)
       time = TimeVal(tv_nsec = 0, tv_sec = 1383923292, tv_usec = 886720)
       type = tango.CmdArgType.DevDouble
      value = array([ -9.61260664e-01,  -9.65924853e-01,  -9.70294813e-01,
        -9.74369212e-01,  -9.78146810e-01,  -9.81626455e-01,
        -9.84807087e-01,  -9.87687739e-01,  -9.90267531e-01,
        ...
        5.15044507e-1])
    w_dim_x = 0
    w_dim_y = 0
w_dimension = AttributeDimension(dim_x = 0, dim_y = 0)
    w_value = None]

Note

due to the internal workings of gevent, setting the wait flag to True (default) doesn’t prevent other greenlets from running in parallel. This is, in fact, one of the major bonus of working with gevent when compared with concurrent.futures

asyncio mode

Asyncio mode is similar to gevent but it uses explicit coroutines. You can compare gevent and asyncio examples.

 1import asyncio
 2from tango.asyncio import DeviceProxy
 3
 4
 5async def asyncio_example():
 6    dev = await DeviceProxy("sys/tg_test/1")
 7    print(dev.get_green_mode())
 8
 9    print(await dev.state())
10
11    # in case of high-level API read has to be awaited
12    print(await dev.long_scalar)
13    print(await dev["long_scalar"])
14    print(await getattr(dev, "long_scalar"))
15
16    # while write executed sync
17    dev.long_scalar = 1
18
19    # for low-level API both read_attribute and write_attribute have to be awaited
20    print(await dev.read_attribute("long_scalar"))
21    await dev.write_attribute("long_scalar", 1)
22
23
24if __name__ == "__main__":
25    asyncio.run(asyncio_example())

Below you can find a TCP server example, which runs in an asynchronous mode and waits for a device’s attribute name from a TCP client, then asks the device for a value and replies to the TCP client.

 1"""A simple TCP server for Tango attributes.
 2
 3It runs on all interfaces on port 8888:
 4
 5   $ python tango_tcp_server.py
 6   Serving on 0.0.0.0 port 8888
 7
 8It can be accessed using netcat:
 9
10   $ ncat localhost 8888
11   >>> sys/tg_test/1/ampli
12   0.0
13   >>> sys/tg_test/1/state
14   RUNNING
15   >>> sys/tg_test/1/nope
16   DevFailed[
17   DevError[
18        desc = Attribute nope is not supported by device sys/tg_test/1
19      origin = AttributeProxy::real_constructor()
20      reason = API_UnsupportedAttribute
21    severity = ERR]
22    ]
23   >>> ...
24"""
25
26import asyncio
27from tango.asyncio import AttributeProxy
28
29
30async def handle_echo(reader, writer):
31    # Write the cursor
32    writer.write(b">>> ")
33    # Loop over client request
34    async for line in reader:
35        request = line.decode().strip()
36        # Get attribute value using asyncio green mode
37        try:
38            proxy = await AttributeProxy(request)
39            attr_value = await proxy.read()
40            reply = str(attr_value.value)
41        # Catch exception if something goes wrong
42        except Exception as exc:
43            reply = str(exc)
44        # Reply to client
45        writer.write(reply.encode() + b"\n" + b">>> ")
46    # Close communication
47    writer.close()
48
49
50async def start_serving():
51    server = await asyncio.start_server(handle_echo, "0.0.0.0", 8888)
52    print("Serving on {} port {}".format(*server.sockets[0].getsockname()))
53    return server
54
55
56async def stop_serving(server):
57    server.close()
58    await server.wait_closed()
59
60
61def main():
62    # Start the server
63    loop = asyncio.get_event_loop()
64    server = loop.run_until_complete(start_serving())
65    # Serve requests until Ctrl+C is pressed
66    try:
67        loop.run_forever()
68    except KeyboardInterrupt:
69        pass
70    # Close the server
71    loop.run_until_complete(stop_serving(server))
72    loop.close()
73
74
75if __name__ == "__main__":
76    main()

Server green modes

PyTango server API from version 9.2.0 supports two green modes: Gevent and Asyncio. Both can be used in writing new device servers in an asynchronous way.

Note

If your device server has multiple devices they must all use the same green mode.

Warning

These green modes disable Tango’s device server serialisation, i.e., tango.SerialModel.NO_SYNC is automatically passed to tango.Util.set_serial_model(), when the device server starts. From those docs: “This is an exotic kind of serialization and should be used with extreme care only with devices which are fully thread safe.”

gevent mode

This mode lets you convert your existing devices to asynchronous devices easily. You just add green_mode = tango.GreenMode.Gevent line to your device class. Consider this example:

class GeventDevice(Device):
    green_mode = tango.GreenMode.Gevent

Every method in your device class will be treated as a coroutine implicitly. This can be beneficial, but also potentially dangerous as it is a lot harder to debug. You should use this green mode with care. Gevent green mode is useful when you don’t want to change too much in your existing code (or you don’t feel comfortable with writing syntax of asynchronous calls).

Another thing to keep in mind is that when using Gevent green mode is that the Tango monitor lock is disabled, so the client requests can be processed concurrently.

Greenlets can also be used to spawn tasks in the background.

asyncio mode

The way asyncio green mode on the server side works is it redirects all user code to an event loop. This means that all user methods become coroutines, so in Python > 3.5 you should define them with async keyword. This also means that in order to convert existing code of your devices to Asyncio green mode you will have to introduce at least those changes. But, of course, to truly benefit from this green mode (and asynchronous approach in general), you should introduce more far-fetched changes!

The main benefit of asynchronous programing approach is that it lets you control precisely when code is run sequentially without interruptions and when control can be given back to the event loop. It’s especially useful if you want to perform some long operations and don’t want to prevent clients from accessing other parts of your device (attributes, in particular). This means that in Asyncio green mode there is no monitor lock!

The example below shows how asyncio can be used to write an asynchronous Tango device:

 1"""Demo Tango Device Server using asyncio green mode"""
 2
 3import asyncio
 4from tango import DevState, GreenMode
 5from tango.server import Device, command, attribute
 6
 7
 8class AsyncioDevice(Device):
 9    green_mode = GreenMode.Asyncio
10
11    async def init_device(self):
12        await super().init_device()
13        self.set_state(DevState.ON)
14
15    @command
16    async def long_running_command(self):
17        self.set_state(DevState.OPEN)
18        await asyncio.sleep(2)
19        self.set_state(DevState.CLOSE)
20
21    @command
22    async def background_task_command(self):
23        loop = asyncio.get_event_loop()
24        future = loop.create_task(self.coroutine_target())
25
26    async def coroutine_target(self):
27        self.set_state(DevState.INSERT)
28        await asyncio.sleep(15)
29        self.set_state(DevState.EXTRACT)
30
31    @attribute
32    async def test_attribute(self):
33        await asyncio.sleep(2)
34        return 42
35
36
37if __name__ == "__main__":
38    AsyncioDevice.run_server()