Preparing release 2.5.0 (#1065)

* namespace fix for napalm cli (#950) * Fix broken get_bgp_neighbors when 6PE configured on eos (#945) * fix _RE_BGP_PREFIX and NEIGHBOR_FILTER for 6PE into eos * add testcase for 6PE version * Renamed testcase mocked file for regexp syntax change * If parent_table None call exception (#939) Hi, i have this situation when ipv6 is turned off: $ napalm --user oxi --password gdfw23 --vendor nxos 10.143.36.8 call get_interfaces_ip 2019-03-07 15:08:10,226 - napalm - ERROR - method - Failed: 'NoneType' object has no attribute 'get' ================= Traceback ================= Traceback (most recent call last): File "/opt/py3.6/bin/napalm", line 10, in <module> sys.exit(main()) File "/opt/py3.6/lib/python3.6/site-packages/napalm/base/clitools/cl_napalm.py", line 309, in main run_tests(args) File "/opt/py3.6/lib/python3.6/site-packages/napalm/base/clitools/cl_napalm.py", line 292, in run_tests call_getter(device, args.method, **method_kwargs) File "/opt/py3.6/lib/python3.6/site-packages/napalm/base/clitools/cl_napalm.py", line 31, in wrapper r = func(*args, **kwargs) File "/opt/py3.6/lib/python3.6/site-packages/napalm/base/clitools/cl_napalm.py", line 256, in call_getter r = func(**kwargs) File "/opt/py3.6/lib/python3.6/site-packages/napalm/nxos/nxos.py", line 1091, in get_interfaces_ip ipv6_command, "TABLE_intf", "ROW_intf" File "/opt/py3.6/lib/python3.6/site-packages/napalm/nxos/nxos.py", line 721, in _get_command_table return self._get_reply_table(json_output, table_name, row_name) File "/opt/py3.6/lib/python3.6/site-packages/napalm/nxos/nxos.py", line 717, in _get_reply_table return self._get_table_rows(result, table_name, row_name) File "/opt/py3.6/lib/python3.6/site-packages/napalm/nxos/nxos.py", line 706, in _get_table_rows _table = parent_table.get(table_name) AttributeError: 'NoneType' object has no attribute 'get' * Remove extensive argument from junos_iface_table (#929) Fixes #928 * Add GitHub issue template, as we used to have before (#961) When we used to maintain separare NAPALM drivers, we had an issue template looking like this: napalm-automation/napalm-junos@110fec4#diff-faa36bc26a21ed93c8de974753b71507. I was looking today through the issues we currently have open, and I found some of them a bit chaotic, and hard to understand the context / environment the user runs. Additionally, some, e.g., #960, is really hard to follow, and I've included a note to invite the user to check the HitGub markdown manual. * NXOS_SSH get_route_to (updated) (#914) * Add MTU support to get_interface (#940) * Feature/ios get bgp config (#897) * support n9k ipv6 ints for interfaces_ip getter (#969) * support catalyst vss for mac_address getter (#968) * support getting auto-negotiated speed on junos (#967) * [IOS] Raise exception if BGP neighbor info requested and BGP not running (#970) * Implement nxos_ssh get_environment (#973) * Switch Docker image to Python 3 and reduce the number of layers and size (#980) * Better handling of connection closing upon exceptions in __enter__ (#994) * Fix incorrect get_interfaces_ip parsing on n9k with IPv6 addresses (#997) * Modernize docs for FreeBSD installation (#993) * Fix nxos and nxos_ssh hostname change handling (#999) * Update link location in the docs (#1002) * Hostname fixes for NXOS and IOS (#1007) * Clean up the documentation contribution section (#1004) * <exception>.message is no longer a thing in Python3 (#1016) * Added option to show run all (#1029) * Update ISSUE_TEMPLATE Removing the space between the brackets, as it seems like it's confusing the users. * junos add optional arg for DB config selection (#1035) * fix junos issue #1028 (#1032) * fix junos issue #1028 * junos add test case for get_network_instances * Incorrectly taken serial number in NXOS. (#1025) * The serial number is taken incorrectly, in particular, the "Processor Board ID" is displayed, and you need a Chassis -> serialnum: * Additional safety net for the lack of a serial number in the output. * black test * mov function _get_table_rows, _get_reply_table and _get_command_table from class NXOSDriver to NXOSDriverBase, because used in NXOSSSHDriver. * Added verification and conversion from string to json * Implemented lock_disable for EOS based on Junos. (#1042) To fix(?) issue #1041 - #1041 On branch eos-lock-disable-flag Changes to be committed: modified: napalm/eos/eos.py * #1025 Work around NXAPI double quoting (#1048) * #1025 Work around NXAPI double quoting The NXAPI on the Nexus 9000 platform seems to be double-quoting the name of inventory items. Amend the check for the inventory item name to take this into account. * the output of the N9000 has extra quotes: show inventory | json {"TABLE_inv": {"ROW_inv": [{"name": "\"Chassis\"", "desc": "\"Nexus9000 C92160YC-X chassis\"",....... * #1046 Use new dictionary key to fetch OS version (#1047) Some Cisco Nexus equipment doesn’t have `sys_ver_str`, but has `rr_sys_ver` instead. Use this as a backup. * Add get_environment support for NXOS driver (#1054) * napalm.nxos.traceroute - VRF traces and handle AS in hops (#989) * Adding traceroute from a specific VRF * Combination of source and vrf command * On some devices running BGP, an AS number will appear after the IP address of the hop * E261 at least two spaces before inline comment [pep8] * Fix extra line in show ip int brief on IOS (#986) * keep nxos consistent with ios for rollback of failed merge (#860) * IOS: add support for VRF for get_arp_table func (#937) Simply takes into account the vrf function arg and exec the expected ios command. * ios get_bgp_neighbors vrf and safi aware (#768) * get_bgp_neighbors vrf support * afi string contains afi modifier * mocked data, minor changes * afi string with afi modifier * supported afis * mocked data * comments * typo * show arp vrf ios test * black reformated * Check if temperature sensor on Cisco device is not supported (#1055) * Check if temperature sensor on Cisco device is not supported Some older Cisco devices like models from the 2960 platform have no temperature sensor built-in. In that case the get_environment function will throw on exception. That case will now be handeled. * Correct if check * Fixes get_environment memory collection on ios-xe devices (#1053) * fixes issue #1052 * add tests for get_environment on ios-xe devices * Pin PyEZ to 2.2.1 (#1061) * get_route_to implementation for IOS (#750) * 1st functional config * bgp protocol attributes added * linting, minor changes * test_get_route_to mocked_data * linting errors fix * comments, minor changes in bgp procesing * bgp processing separated * bgp support 7200 old * new bgp processing * mocked data global routing table * comments, _get_vrfs update * mocked data c7200 * bug in regex * connected, next-hop regex changed,.. * community + ext.community, comments * linting * RE_RDB1 regex fixed * asr_pe test * typo * typo * typo in mocked data - vrf Test -> TEST * longer parameter added (PR735) * rollback to pass test_method_signatures ... * bgp_time_conversion duplicity fixed * community regex fixed * code optimization * module constant, variables renaming * black reformated * sh arp vrf test * When asPathType is Internal (IBGP), as_path is empty and the remote-as is the local_as (#1064) * Pin Netmiko to version 2.4.2 (#1066) * Release 2.5.0 (#1063)
napalm-automation · Oct 10, 2019 · a65fc25 · a65fc25
1 parent 2332653
commit a65fc25
Show file tree

Hide file tree

Showing 157 changed files with 28,366 additions and 2,286 deletions.
diff --git a/.github/ISSUE_TEMPLATE b/.github/ISSUE_TEMPLATE
@@ -0,0 +1,36 @@
+### Description of Issue/Question
+
+*Note*: Please check https://guides.github.com/features/mastering-markdown/
+to see how to properly format your request.
+
+### Did you follow the steps from https://github.com/napalm-automation/napalm#faq
+(Place an ``x`` between the square brackets where applicable)
+
+- [] Yes
+- [] No
+
+
+### Setup
+
+### napalm version
+(Paste verbatim output from `pip freeze | grep napalm` between quotes below)
+
+```
+
+```
+
+### Network operating system version
+(Paste verbatim output from `show version` - or equivalent - between quotes below)
+
+```
+
+```
+
+### Steps to Reproduce the Issue
+
+### Error Traceback
+(Paste the complete traceback of the exception between quotes below)
+
+```
+
+```
diff --git a/Dockerfile b/Dockerfile
@@ -1,15 +1,10 @@
-FROM debian:stretch
-
-## Install min deps
-RUN apt-get update
+FROM python:3.6-slim-stretch
 
 COPY ./ /var/cache/napalm/
 
-## Install NAPALM & underlying libraries dependencies
-RUN apt-get install -y python-cffi python-dev libxslt1-dev libssl-dev libffi-dev \
-    && apt-get install -y python-pip \
-    && pip install -U cffi \
-    && pip install -U cryptography \
-    && pip install /var/cache/napalm/
-
-RUN rm -rf /var/lib/apt/lists/*
+RUN apt-get update \
+ && apt-get install -y python-dev python-cffi libxslt1-dev libssl-dev libffi-dev \
+ && apt-get autoremove \
+ && rm -rf /var/lib/apt/lists/* \
+ && pip --no-cache-dir install -U cffi cryptography /var/cache/napalm/ \
+ && rm -rf /var/cache/napalm/
diff --git a/docs/contributing/index.rst b/docs/contributing/index.rst
@@ -44,7 +44,27 @@ If you found a bug and know how to fix just contribute the bugfix. It might be i
 Documentation
 -------------
 
-Just do it! :)
+The documentation is built using `Sphinx`_ and hosted on Read the Docs. The docs
+are kept in the ``docs/`` directory at the top of the source tree.
+
+To make changes, it is preferable to set up a `python virtual environment`_
+called ``env`` and activate it.
+
+Next, install the documentation dependencies using ``pip``:
+
+.. code:: console
+
+   pip install -r docs/requirements.txt
+
+Make your changes and then check them for correctness by building them locally:
+
+.. code:: console
+
+   # in the docs directory
+   make html
+
+.. tip:: If it is a simple change to a single page, you can use the "Edit on
+   GitHub" button in the upper right hand corner of the page in question.
 
 Proposing a new driver
 ----------------------
@@ -58,3 +78,6 @@ Please check :ref:`contributing-drivers` to understand the process.
 
    core
    drivers
+
+.. _python virtual environment: https://packaging.python.org/guides/installing-using-pip-and-virtualenv/
+.. _Sphinx: http://www.sphinx-doc.org/en/master/
diff --git a/docs/installation/index.rst b/docs/installation/index.rst
@@ -14,6 +14,25 @@ You can install napalm with pip:
 That will install all the drivers currently available.
 
 
+OS Package Managers
+-------------------
+
+Some execution environments offer napalm through a system-level package manager. Installing with pip outside of a user profile or virtualenv/venv is inadvisable in these cases.
+
+FreeBSD
+~~~~~~~
+
+.. code-block:: bash
+
+    pkg install net-mgmt/py-napalm
+
+This will install napalm and all drivers and dependencies for the default version(s) of python. To install for a specific version, python X.Y, if supported:
+
+.. code-block:: bash
+
+    pkg install pyXY-napalm
+
+
 Dependencies
 ------------
 

diff --git a/docs/installation/ios.rst b/docs/installation/ios.rst
@@ -15,11 +15,3 @@ RedHat and CentOS
 .. code-block:: bash
 
     sudo yum install -y python-pip gcc openssl openssl-devel libffi-devel python-devel
-
-
-FreeBSD
--------
-
-.. code-block:: bash
-
-    sudo pkg_add -r py27-pip
diff --git a/docs/installation/iosxr.rst b/docs/installation/iosxr.rst
@@ -15,11 +15,3 @@ RedHat and CentOS
 .. code-block:: bash
 
     sudo yum install -y python-pip gcc openssl openssl-devel libffi-devel python-devel
-
-
-FreeBSD
--------
-
-.. code-block:: bash
-
-    sudo pkg_add -r py27-pip
diff --git a/docs/installation/junos.rst b/docs/installation/junos.rst
@@ -15,10 +15,3 @@ RedHat and CentOS
 .. code-block:: bash
 
     sudo yum install -y python-pip python-devel libxml2-devel libxslt-devel gcc openssl openssl-devel libffi-devel
-
-FreeBSD
--------
-
-.. code-block:: bash
-
-    sudo pkg_add -r py27-pip libxml2 libxslt
diff --git a/docs/validate/index.rst b/docs/validate/index.rst
@@ -263,7 +263,7 @@ CLI & Ansible
 If you prefer, you can also make use of the validate functionality via the CLI with the command ``cl_napalm_validate`` or with ansible plugin. You can find more information about them here:
 
 * CLI - https://github.com/napalm-automation/napalm/pull/168
-* Ansible - https://github.com/napalm-automation/napalm-ansible/blob/master/library/napalm_validate.py
+* Ansible - https://github.com/napalm-automation/napalm-ansible/blob/master/napalm_ansible/modules/napalm_validate.py
 
 
 Why this and what's next

diff --git a/napalm/base/base.py b/napalm/base/base.py
@@ -16,14 +16,16 @@
 from __future__ import print_function
 from __future__ import unicode_literals
 
+import sys
+
+from netmiko import ConnectHandler, NetMikoTimeoutException
+
 # local modules
 import napalm.base.exceptions
-from napalm.base.exceptions import ConnectionException
 import napalm.base.helpers
 from napalm.base import constants as c
 from napalm.base import validate
-
-from netmiko import ConnectHandler, NetMikoTimeoutException
+from napalm.base.exceptions import ConnectionException
 
 
 class NetworkDriver(object):
@@ -44,8 +46,15 @@ def __init__(self, hostname, username, password, timeout=60, optional_args=None)
         raise NotImplementedError
 
     def __enter__(self):
-        self.open()
-        return self
+        try:
+            self.open()
+            return self
+        except:  # noqa: E722
+            # Swallow exception if __exit__ returns a True value
+            if self.__exit__(*sys.exc_info()):
+                pass
+            else:
+                raise
 
     def __exit__(self, exc_type, exc_value, exc_traceback):
         self.close()
@@ -95,8 +104,9 @@ def _netmiko_open(self, device_type, netmiko_optional_args=None):
 
     def _netmiko_close(self):
         """Standardized method of closing a Netmiko connection."""
-        self.device.disconnect()
-        self._netmiko_device = None
+        if getattr(self, "_netmiko_device", None):
+            self._netmiko_device.disconnect()
+            self._netmiko_device = None
         self.device = None
 
     def open(self):
@@ -268,6 +278,7 @@ def get_interfaces(self):
          * description (string)
          * last_flapped (float in seconds)
          * speed (int in Mbit)
+         * MTU (in Bytes)
          * mac_address (string)
 
         Example::
@@ -280,6 +291,7 @@ def get_interfaces(self):
                 'description': '',
                 'last_flapped': -1.0,
                 'speed': 1000,
+                'mtu': 1500,
                 'mac_address': 'FA:16:3E:57:33:61',
                 },
             u'Ethernet1':
@@ -289,6 +301,7 @@ def get_interfaces(self):
                 'description': 'foo',
                 'last_flapped': 1429978575.1554043,
                 'speed': 1000,
+                'mtu': 1500,
                 'mac_address': 'FA:16:3E:57:33:62',
                 },
             u'Ethernet2':
@@ -298,6 +311,7 @@ def get_interfaces(self):
                 'description': 'bla',
                 'last_flapped': 1429978575.1555667,
                 'speed': 1000,
+                'mtu': 1500,
                 'mac_address': 'FA:16:3E:57:33:63',
                 },
             u'Ethernet3':
@@ -307,6 +321,7 @@ def get_interfaces(self):
                 'description': 'bar',
                 'last_flapped': -1.0,
                 'speed': 1000,
+                'mtu': 1500,
                 'mac_address': 'FA:16:3E:57:33:64',
                 }
             }
@@ -1497,13 +1512,14 @@ def get_optics(self):
         """
         raise NotImplementedError
 
-    def get_config(self, retrieve="all"):
+    def get_config(self, retrieve="all", full=False):
         """
         Return the configuration of a device.
 
         Args:
             retrieve(string): Which configuration type you want to populate, default is all of them.
                               The rest will be set to "".
+            full(bool): Retrieve all the configuration. For instance, on ios, "sh run all".
 
         Returns:
           The object returned is a dictionary with a key for each configuration store:

diff --git a/napalm/base/clitools/cl_napalm.py b/napalm/base/clitools/cl_napalm.py
@@ -146,6 +146,9 @@ def build_help():
     )
     args = parser.parse_args()
 
+    if not hasattr(args, "which"):
+        args.which = None
+
     if args.password is None:
         password = getpass.getpass("Enter password: ")
         setattr(args, "password", password)

diff --git a/napalm/base/helpers.py b/napalm/base/helpers.py
@@ -6,6 +6,7 @@
 
 # std libs
 import os
+import re
 import sys
 import itertools
 
@@ -15,6 +16,7 @@
 from netaddr import EUI
 from netaddr import mac_unix
 from netaddr import IPAddress
+from ciscoconfparse import CiscoConfParse
 
 # local modules
 import napalm.base.exceptions
@@ -108,6 +110,67 @@ def load_template(
     return cls.load_merge_candidate(config=configuration)
 
 
+def cisco_conf_parse_parents(parent, child, config):
+    """
+    Use CiscoConfParse to find parent lines that contain a specific child line.
+
+    :param parent: The parent line to search for
+    :param child:  The child line required under the given parent
+    :param config: The device running/startup config
+    """
+    if type(config) == str:
+        config = config.splitlines()
+    parse = CiscoConfParse(config)
+    cfg_obj = parse.find_parents_w_child(parent, child)
+    return cfg_obj
+
+
+def cisco_conf_parse_objects(cfg_section, config):
+    """
+    Use CiscoConfParse to find and return a section of Cisco IOS config.
+    Similar to "show run | section <cfg_section>"
+
+    :param cfg_section: The section of the config to return eg. "router bgp"
+    :param config: The running/startup config of the device to parse
+    """
+    return_config = []
+    if type(config) is str:
+        config = config.splitlines()
+    parse = CiscoConfParse(config)
+    cfg_obj = parse.find_objects(cfg_section)
+    for parent in cfg_obj:
+        return_config.append(parent.text)
+        for child in parent.all_children:
+            return_config.append(child.text)
+    return return_config
+
+
+def regex_find_txt(pattern, text, default=""):
+    """""
+    RegEx search for pattern in text. Will try to match the data type of the "default" value
+    or return the default value if no match is found.
+    This is to parse IOS config like below:
+    regex_find_txt(r"remote-as (65000)", "neighbor 10.0.0.1 remote-as 65000", default=0)
+    RETURNS: 65001
+
+    :param pattern: RegEx pattern to match on
+    :param text: String of text ot search for "pattern" in
+    :param default="": Default value and type to return on error
+    """
+    text = str(text)
+    value = re.findall(pattern, text)
+    try:
+        if not value:
+            raise Exception
+        if not isinstance(value, type(default)):
+            if isinstance(value, list) and len(value) == 1:
+                value = value[0]
+            value = type(default)(value)
+    except Exception:  # in case of any exception, returns default
+        value = default
+    return value
+
+
 def textfsm_extractor(cls, template_name, raw_text):
     """
     Applies a TextFSM template over a raw text and return the matching table.

diff --git a/napalm/base/test/models.py b/napalm/base/test/models.py
@@ -19,6 +19,7 @@
     "is_enabled": bool,
     "description": text_type,
     "last_flapped": float,
+    "mtu": int,
     "speed": int,
     "mac_address": text_type,
 }