Shells logging standard

[Quali-Community]

As part of the Shell Standardization initiative the way the Shell logs are structured has been standardize and a logger that can be used by any Shell was created.

The Networking Shell Standard can be found here.

Definition

Logs Structure

The logs folder is structured as follows:

• reservationId
• <ResourceName>--DD-MM-YYYY--HH-mm-SS.log
• <EnvironmentName>--DD-MM-YYYY--HH-mm-SS.log
• Autoload
• <ResourceName>--DD-MM-YYYY--HH-mm-SS.log

There is a folder per each reservation and its name is equal to the reservation ID. For Autoload commands that are being executed outside of the reservation context there is a separate folder called Autoload.

In those folders there is a file per each resource. The file name is resourceName--D-MMM-YYYY--HH-MM-SS(the time in the file indicates the creation time of the file). In case of environment commands that run as part of an environment orchestration a separate file will be created, this file is namedenvironmentName--D-MMM-YYYY--HH-MM-SS.logand it will include logs both for environment commands (which are part of a driver) and environments scripts.

Each log entry contains the following – date, time, verbosity level, module name (max 50 characters), method name (max 50 characters) and message.

Description of the log record attributes: https://docs.python.org/2/library/logging.html#logrecord-attributes

Logs Configuration

The logger is located in cloudshell-core Python package (link to repository) and comes with a configuration in which it is possible to customize the logs.

To customize the logs verbosity change the LOG_LEVEL from DEBUG to INFO, WARN or ERROR.

It is also possible to customize the structure of a log entry in the LOG_FORMAT section, and the time format used in each log entry in the TIME_FORMAT section.

The location in which the logs will be located is defined in the LOG_PATH.

The default location “../../Logs” translates to the following path in 6.4 Shells:

C:\ProgramData\QualiSystems\QsPython27\Lib\site-pacakges\cloudshell\Logs

Starting from 7.0, the Shells use the python driver infrastructure and the default location “../../Logs” translates to the following path:

C:\ProgramData\QualiSystems\venv\{driverName_version}\Lib\site-packages\cloudshell\Logs

This location refers to a location on each execution server. It is possible to change the path to ashared location that is accessible (including write permissions) to the execution server. To change that just update the LOG_PATH in the qs_config.ini file to‘{sharedLocation}:/Logs/’and restart the execution server.

How to use the standard Logger

Setup the cloudshell-core package:

To use qs_logger install attached cloudshell-core-1.0.11.zip package:

1. Run ‘cmd’ as Administrator
2. cd [Python installation path]\Scripts (in your case it can be C:\ProgramData\QualiSystems\QsPython27\Scripts)
3. run pip install to install the cloudshell-core

Logger example script:

How to use the standard Logger in a Networking Shell

The networking shell that follows the standard has the following python packages structure:

cloudshell-core

--> cloudshell-shell-core

--> cloudshell-networking

--> cloudshell-networking-[vendor]

--> cloudshell-networking-[vendor]-[os]

The logger described in the previous section is located in the “cloudshell-core” and all the definitions required for using it are located in the networking generic resource driver ( cloudshell-networking/cloudshell/networking/resource_driver/networking_generic_resource_driver.py). Any resource driver that will be based on the networking generic resource driver will get all the logger definitions.

For a resource driver to be based on the networking generic resource driver it should inherit the networking_generic_resource_driver class and use this class’ Init function in the new driver’s Init:

Once the above is defined it is possible to write to the logger in the following format:

Replace “info” with the desired verbosity level (debug, info, warn, error).

from qs_logger import *
def log_something(m_logger, message):
    m_logger.info(message)
    if __name__ == '__main__':
	# test_logger = get_qs_logger('logger_name', 'resource_name', 'reservation_id')
    test_logger = get_qs_logger()
    log_something(test_logger,'>>>My Message<<<')
    test_logger.debug('Sending Command to device')
    test_logger.info('This is Info message')
    test_logger.warning('This is Warning message')
    test_logger.error('This is Critical error message!')
    print 'OK'

 Attachments:

1.png [14KB]	2.png [26KB]

---

 Alona Getzler (Alona.G) - 06/05/2016 12:26 PM

·  180  ·

[Quali-Community]

The use of virtual environments for resource driver execution complicates this. It means that logs for each driver are in different locations (under the site-packages for the associated venv). AND it means that the configuration file has to be set for each resource driver separate - AND it can only be set AFTER you've run once and the venv has been created, AND it gets reset back to default every time you modify the driver which triggers a new venv to get created.

---

Gregory Sloan (Gregory.S) - 07/22/2016 10:45 PM

[Quali-Community]

Hi Greg,

One way to overcome these issues, is by setting a default log path for all the drivers in an execution server.
You can do that by adding the following key to the ES customer.config (as explained in the dev guide):

<add key="DefaultPythonEnvrionmentVariables" value="LOG_PATH=c:\quali\logs"/>

The value is in a csv format, you can set other parameters that will be available in the os.environ later on.
One of them is also the default log level:

<add key="DefaultPythonEnvrionmentVariables" value="LOG_PATH=c:\quali\logs,LOG_LEVEL=INFO,MYVAR=value"/>

---

Yaniv Kalsky (Yaniv.K) - 08/29/2016 05:19 PM

[Quali-Community]

That is not in the DevGuide and incredibly useful. Can that be made front and center?

---

Chris Grabosky (Chris.G) - 10/10/2016 01:17 PM

[Quali-Community]

Good point. Ricardo just posted an article on making a pull request to update the Dev Guide, so that can easily be added. search:?discussions_q=request+to+update

---

Pascal Joly (Pascal.J) - 10/20/2016 12:24 AM