Skip to content

Commit

Permalink
SONiC Management Framework Release 1.0 (#18)
Browse files Browse the repository at this point in the history
* [JIRA SONIC-9259] Add range limit to mtu in sonic-port.

Change-Id: I5917eb03c44410fcf0aa0ceb5720cd5c12415d00

* Signature chnage for transformer functions

* modified acl-key xfmr to return integer and enhanced the get-req handling in common app.

* make common-app as default to handle acl, add the field-xfmr for acl type

* [JIRA SONIC-9203] Added IP_TYPE field with value ANY in DEFAULT_RULE.

Change-Id: I82ec0a1610da009f9906d0ac2ab7c5eb5aa4b13b

* Fix for bugs 96, 97 and 107

* Addressed the comment to sort the speed

* Fix sonic get query failure

* fix to exclude key-leaf data populating db-filed value and removed commented lines in xmfr_acl

* Change MAX_MTU to 9216 in platform_dummy file as well, since it
overwrites it.

* Fix for bugs 96, 97 and 107 (Speed is not displaying properly) (#109)

* Fix for not showing description when description is configured as part of #show interface command.

* Addressing the crash when max mtu is set as 9276. Currently supported max-mtu is 9216.

* Fix for bugs 96, 97 and 107

* Addressed the comment to sort the speed

* Change MAX_MTU to 9216 in platform_dummy file as well, since it
overwrites it.

* fix for show process not working (#110)

* added unit8 yang data0type support for get-req

* Fix crash for get leaf query

* cleanup code to showcase overloaded methods

* [JIRA SONIC-9466] merge mgmt-framework bug fixes from Dell - 12th Sept

Change-Id: I2a92251040a816c24f8433e49dd508091dbb494b

* fix the build error

* Fis get leaf crash, remove unsed annotations

* [JIRA SONIC-9426] CVL throws error when trying to configure  IP address.

Change-Id: Iec675bff98c912ef5d80a2d38524748d6a0b1683

* [JIRA SONIC-9494] CVL Error seen when performing  no description  on interface

Change-Id: Ic3e89d74bd399e891a06e14bd4ff7dcd486a9090

* [JIRA SONIC-9420] Updated Help string to display max length of ACL is 63
characters. Added pattern and length check in CVL yang also to allow 72
characters (63 of acl name + 9 from acl type ex. _ACL_IPV4)

Change-Id: Iba9ffc7aa34f1600c61b95ec9a3e23971533e829

* [JIRA SONIC-9496] Disabled nonyang demo app

Disabled nonyang vlan demo app by adding .demo suffix to yaml and app
module code. It can bemanually enabled by renaming those files back.

Change-Id: I1714e772fb452ae246d44af65d43dd1cf65f5e18

* Adding Interface table entry with just name, before the one with ifname
and IP.

* add get-validate to validate a YANG node durin data translation back to YANG response

* Fix crash during update request to leaflist

* Fix top level delete

* Fix: Incorrect LLDP neighbor TTL (#117)

* Remove the TTL information from CLI and GNMI output because
  LLDP neighbor TTL information is currently not stored in appDB.

Signed-off-by: Garrick He <[email protected]>

* Bugfix for creating 2 interface entries when IP is configured, and when interface comes up as L2. (#116)

* [JIRA SONIC-9314] Modified the MAX_PRIORITY constant  to 65536, to help app module
to translate between SeqId and priority when given max seqid of 65535.

Change-Id: I314e5cabb89779e0410d05d18152425f2c0228b2

* [JIRA SONIC-9472] Modified logic in Jinja template for "show ip
access-group" to display both Ingress and Egress ACLs for single port.
Also updated help message for max allowed characters for acl name in
"show" commands.

sonic# show ip access-group
Ingress IP access-list A1 on Ethernet0
Egress IP access-list A2 on Ethernet0
Ingress IP access-list A1 on Ethernet4
Egress IP access-list A2 on Ethernet4
sonic#

Change-Id: I601e1b21b01e40ea6a285400e803092c66f1c941

* [JIRA SONIC-9521]  Merge mgmt-framework changes by Dell team - 13th Sep

Change-Id: I2a830e2198b506525ae970377a5bbfb49a98006b

* [JIRA SONIC-9486] Permit only TLS1.2 or higher in REST Server

Management REST Server is enhanced to use only TLS1.2 or higer version
with one of below cipher suites (in order).
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Similar changes to gNMI telemetry server will be done by Dell team.

UT:
1) Verified ACL CLIs, which also verifies client SDK
2) Verified swagger UI with Chrome and Firefox browsers
3) Verified thru curl that sslv2, sslv3, tlsv1.0, tlsv1.1 clients are
rejected by REST Server.
4) Verified thru curl that unsupported ciphers
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA and TLS_RSA_WITH_3DES_EDE_CBC_SHA
are rejected by REST Server.
5) Mgmt-framework automated UT

Change-Id: I378522335f69d9a73ff7d62dc7ceec79087faf2f

* added yang annotations

* Get case - leaf-list support in Transformer core for Sonic and OC yang.

* add the field transformer for forwarding-action

* Set case - leaf-list support in transformer core for OC yang

* provision to get data for sibling container from the module level

* Fixed variable scope issue while processing leaf data to  redis DB
format

* fix license header

* Updated license statement for REST server sources

* Update license header

* update broadcom license for pyang plugin for openAPI

* Updated License statement

* Update broadcom license

* DB Access Layer: Update Broadcom License

* Add validate callback function support for acl. Children will inherit parent validate handler

* Add copyright/license headers (#123)

Signed-off-by: Tejaswi Goel <[email protected]>

* move json creation for sonic-yang to map

* Moving unwanted CVL YANG to testdata

* Updated Makefiles

* Delete file checked in mistakenly.

* Updated new Makefile with license, copy schema from testdata/schema also.

* Updated License statement

* Update License statement

* update broadcom license and made changes to keep the diff file in the ygot-modified directory

* error handling and code clean-up

* update broadcom license to the diff files which are created for patching ygot files

* Add error handling in xlate_utils.go

* fix the uint64 encoding, fix the ygot bug in handling the union type of
leafref

* For a Get req on individual terminal node- Fix overloaded function processing, data-type conversion and leaf-list
check handling

* table transformer support

* CLI skeletion for VLAN feature: config, show and its templates

* Made changes to keep the patch files into single patch file

* Cleanup of unused code

* Updated the ygot patch file

* Changes for supporting show command with dummy data

* add the union type

* Update Makefile

Add rule to compile test YANG schema

* CLI skeleton for Port channel

Signed-off-by: Tejaswi Goel <[email protected]>

* Add post transformer infra

* CVL changes for SONiC YANG

* CVL changes for SONiC YANG

* Initial support for table transformer

* Fix issues in initial commit for table transformer

* copy the misisng lines from transformer-phase1

* add missing lines from transforer-phase1

* support for table transformer for crud and get operation

* Extend annotations for augmentations in oc-yang

* Addressing the comments mentioned

* Fixed alignment of XML files

* Cleanup of the xml files

* Handle nested groupings for augmentation

* changed the xSpecMap to xYangSpecMap for code clarity and added new file with transformer-constants

* Changes to remove dummy json file

Signed-off-by: Tejaswi Goel <[email protected]>

* Enhanced transformer core to support user-defined key-delimiter/concatenater

* basic support for sonic-yang annotation

* Fill cvl KeySpec with DbName read from annotation

* More changes for CVL YANG

* Fix table get in Traverse for App DB with ":" separator

* added support to invoke container level key transformer

* added code to pass the current ygot node to the key xfmr function

* added support for sonic yang non-config db get request

* Data base specific key-delimiter support in transformer core - Sonic Get
case

* Changes in show port-channel command

Signed-off-by: Tejaswi Goel <[email protected]>

* Add checks for xpath entry in xYangSpecMap and xDbSpecMap

* Chk for list type to create keySpec

* workaround for VLAN_MEMBER_TABLE list level get due to translib.generateGetResponsePayload()  failure

* added key-name support for oc yang and support to create and push table with just the keys and no field-value data in to redis.

* Updated Makefile

* OC Yang Get case - DB type based key-delimiter support in transformer
core

* CVl changes for SONiC YANG

* CVL changes for SONiC YANG

* Add check for no xfmrFunc defined case

* Correct naming convention for Interface commands

Signed-off-by: Tejaswi Goel <[email protected]>

* Changed processLeafList() function name to checkAndProcessLeafList() , added comments and removed unnecessary logs in common app

* fix for vlan get-req crash

* sonic yang get case handling for table instance level

* Handle nested db-name annotations for oc

* Import module name of the annotated module in annotate template file

* Added table transformer method prototype.

* Added table transformer extension

* Include the prefix for the module where the grouping is used in annotation template

* Include unique namespace, Use the prefix of the including module

* xspec map and cru support in transformer infra per revised sonic-yang and changed the api-names from cvl to sonic

* Management interface cli tree, actioner implementation and sonic yang  for management port and interface config handling. (#153)

* sFlow CLI Template (#147)

* sFlow CLI template

* Add XML skeleton for the sFlow CLI
* Add actioner script
* Add renderer template

Signed-off-by: Garrick He <[email protected]>

* added code to support reading db data for get req per revised sonic-yang

* rearranged the code to call the table transformer and collect data before calling the subtree-transformer overloaded method.

* Fixed table transformer GET case issues :
   1.) calling table-transformer for each leaf when doing a get on parent conatiner(OC-interface)
   2.) dbMapData not getting filled for container query in dbDataToYangJsonCreate

* Handle delete for restructured sonic yang

* Strip augmented module names in xpath to reference xYangSpecMap

* transformer infra support for revised sonic-yang GET req

* Handle delete for restructured sonic acl

* Fix the leaf level get for restructured sonic yang

* Make file changes to generate sonic yaml files

* support table transformer - GET case on leaf or nested container

* Get for sonic yang at instance level to return single instance data

* Use new xpathPredicate function that removes augmented module name in the uri

* Bug fixed for not showing GB instead of Gx

* Add Copyright info

* Sonic yang restructuring fix terminal node leaf/leaf-list cases

* revert accidently added local changes in xspec.go

* Clean up of files and add missed items during merge

* exclude annotation file from generating ocbinds

* fix Makefile in installing yang files from cv/schema

* add back the cv/schema/*yin

* Fixed list instance level GET issue

* Revert "Merge pull request #173 from project-arlo/transformer-phase2"

This reverts commit ca4b330, reversing
changes made to 211f17e.

* Revert "Merge branch 'transformer-phase1' into transformer_merge"

This reverts commit 211f17e, reversing
changes made to f032ec5.

* commit Transformer codes for community PR

* Update Makefile

* Update Makefile

* undo ygot patch changes done by kwan

* update the patch file

* Patch goyang files to make rules

* Add missing models list file

* Remove network-instance yang

* Addressing PR review comments.

*  Add new file entry in goyang.patch file

* Incorporating review comments - phase2

* Adding comment for TRACE_ONERROR

* Moving go-server.sh to tools/test/rest-server.sh

* Correcting regex pattern for key names

* Correct error code if key does not exist

* Address review comments

* Address review comments

* Address PR review comments - phase2

* Add validation for speed and description (#291)

* Add validation for speed and description

* Add pattern for alias

* Updated the Makefile to have go pkg version, and renamed the ACL CLI actioner file

* Update control

* Moving jsonquery patch in patches/jsonquery.patch file, updating
testcases as per latest YANG model.

* Minor change in test case.

* Updating makefile

* Checkout jsonquery only once.

* Use git clean before patching to ensure no files are left behind

Co-authored-by: asha-behera <[email protected]>
Co-authored-by: ranjinidn <[email protected]>
Co-authored-by: s-mari <[email protected]>
Co-authored-by: Kwan <[email protected]>
Co-authored-by: Mayank Maheshwari <[email protected]>
Co-authored-by: Justine Jose <[email protected]>
Co-authored-by: Arunsundar Kannan <[email protected]>
Co-authored-by: Kwan Kim <[email protected]>
Co-authored-by: Sachin Holla <[email protected]>
Co-authored-by: Garrick He <[email protected]>
Co-authored-by: Partha Dutta <[email protected]>
Co-authored-by: amrutasali <[email protected]>
Co-authored-by: Faraaz mohammed <[email protected]>
Co-authored-by: anand-kumar-subramanian <[email protected]>
Co-authored-by: a-barboza <[email protected]>
Co-authored-by: Tejaswi Goel <[email protected]>
Co-authored-by: Balachandar Mani <[email protected]>
Co-authored-by: rvasanthm <[email protected]>
Co-authored-by: joyas-joseph <[email protected]>
Co-authored-by: Eric Seifert <[email protected]>
  • Loading branch information
21 people authored and renukamanavalan committed Dec 23, 2019
1 parent 0755674 commit 8b199a9
Show file tree
Hide file tree
Showing 315 changed files with 65,396 additions and 0 deletions.
15 changes: 15 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@

.vscode
*.code-workspace
pkg/*
build
gopkgs
__pycache__
*.pyc
*.rdb
*.swp

*.yin
*.tree
src/translib/ocbinds/ocbinds.go

166 changes: 166 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
################################################################################
# #
# Copyright 2019 Broadcom. The term Broadcom refers to Broadcom Inc. and/or #
# its subsidiaries. #
# #
# Licensed under the Apache License, Version 2.0 (the "License"); #
# you may not use this file except in compliance with the License. #
# You may obtain a copy of the License at #
# #
# http://www.apache.org/licenses/LICENSE-2.0 #
# #
# Unless required by applicable law or agreed to in writing, software #
# distributed under the License is distributed on an "AS IS" BASIS, #
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #
# See the License for the specific language governing permissions and #
# limitations under the License. #
# #
################################################################################

.PHONY: all clean cleanall codegen rest-server rest-clean yamlGen cli

TOPDIR := $(abspath .)
BUILD_DIR := $(TOPDIR)/build
export TOPDIR

ifeq ($(BUILD_GOPATH),)
export BUILD_GOPATH=$(TOPDIR)/gopkgs
endif

export GOPATH=$(BUILD_GOPATH):$(TOPDIR)

ifeq ($(GO),)
GO := /usr/local/go/bin/go
export GO
endif

INSTALL := /usr/bin/install

MAIN_TARGET = sonic-mgmt-framework_1.0-01_amd64.deb

GO_DEPS_LIST = github.com/gorilla/mux \
github.com/Workiva/go-datastructures/queue \
github.com/openconfig/goyang \
github.com/openconfig/ygot/ygot \
github.com/go-redis/redis \
github.com/golang/glog \
github.com/pkg/profile \
gopkg.in/go-playground/validator.v9 \
golang.org/x/crypto/ssh \
github.com/antchfx/jsonquery \
github.com/antchfx/xmlquery \
github.com/facette/natsort \
github.com/philopon/go-toposort


REST_BIN = $(BUILD_DIR)/rest_server/main
CERTGEN_BIN = $(BUILD_DIR)/rest_server/generate_cert


all: build-deps go-deps go-pkg-version go-patch translib rest-server cli

build-deps:
mkdir -p $(BUILD_DIR)

go-deps: $(GO_DEPS_LIST)

go-pkg-version: go-deps
cd $(BUILD_GOPATH)/src/github.com/go-redis/redis; git checkout d19aba07b47683ef19378c4a4d43959672b7cec8 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/go-redis/redis; \
cd $(BUILD_GOPATH)/src/github.com/gorilla/mux; git checkout 49c01487a141b49f8ffe06277f3dca3ee80a55fa 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/gorilla/mux; \
cd $(BUILD_GOPATH)/src/github.com/Workiva/go-datastructures; git checkout f07cbe3f82ca2fd6e5ab94afce65fe43319f675f 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/Workiva/go-datastructures; \
cd $(BUILD_GOPATH)/src/github.com/golang/glog; git checkout 23def4e6c14b4da8ac2ed8007337bc5eb5007998 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/golang/glog; \
cd $(BUILD_GOPATH)/src/github.com/pkg/profile; git checkout acd64d450fd45fb2afa41f833f3788c8a7797219 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/pkg/profile; \
cd $(BUILD_GOPATH)/src/github.com/antchfx/xmlquery; git checkout 16f1e6cdc5fe44a7f8e2a8c9faf659a1b3a8fd9b 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/antchfx/xmlquery; \
cd $(BUILD_GOPATH)/src/github.com/facette/natsort; git checkout 2cd4dd1e2dcba4d85d6d3ead4adf4cfd2b70caf2 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/facette/natsort; \
cd $(BUILD_GOPATH)/src/github.com/philopon/go-toposort; git checkout 9be86dbd762f98b5b9a4eca110a3f40ef31d0375 2>/dev/null ; true; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/philopon/go-toposort

$(GO_DEPS_LIST):
$(GO) get -v $@

cli: rest-server
$(MAKE) -C src/CLI

cvl: go-deps go-patch go-pkg-version
$(MAKE) -C src/cvl
$(MAKE) -C src/cvl/schema
$(MAKE) -C src/cvl/testdata/schema

cvl-test:
$(MAKE) -C src/cvl gotest

rest-server: translib
$(MAKE) -C src/rest

rest-clean:
$(MAKE) -C src/rest clean

translib: cvl
$(MAKE) -C src/translib

codegen:
$(MAKE) -C models

yamlGen:
$(MAKE) -C models/yang
$(MAKE) -C models/yang/sonic

go-patch: go-deps
cd $(BUILD_GOPATH)/src/github.com/openconfig/ygot/; git reset --hard HEAD; git clean -f -d; git checkout 724a6b18a9224343ef04fe49199dfb6020ce132a 2>/dev/null ; true; \
cd ../; cp $(TOPDIR)/ygot-modified-files/ygot.patch .; \
patch -p1 < ygot.patch; rm -f ygot.patch; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/openconfig/ygot/ygot; \
cd $(BUILD_GOPATH)/src/github.com/openconfig/goyang/; git reset --hard HEAD; git clean -f -d; git checkout 064f9690516f4f72db189f4690b84622c13b7296 >/dev/null ; true; \
cp $(TOPDIR)/goyang-modified-files/goyang.patch .; \
patch -p1 < goyang.patch; rm -f goyang.patch; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/openconfig/goyang; \
cd $(BUILD_GOPATH)/src/github.com/antchfx/jsonquery; git reset --hard HEAD; \
git checkout 3535127d6ca5885dbf650204eb08eabf8374a274 2>/dev/null ; \
git apply $(TOPDIR)/patches/jsonquery.patch; \
$(GO) install -v -gcflags "-N -l" $(BUILD_GOPATH)/src/github.com/antchfx/jsonquery

install:
$(INSTALL) -D $(REST_BIN) $(DESTDIR)/usr/sbin/rest_server
$(INSTALL) -D $(CERTGEN_BIN) $(DESTDIR)/usr/sbin/generate_cert
$(INSTALL) -d $(DESTDIR)/usr/sbin/schema/
$(INSTALL) -d $(DESTDIR)/usr/sbin/lib/
$(INSTALL) -d $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/models/yang/sonic/*.yang $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/models/yang/sonic/common/*.yang $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/src/cvl/schema/*.yin $(DESTDIR)/usr/sbin/schema/
$(INSTALL) -D $(TOPDIR)/src/cvl/testdata/schema/*.yin $(DESTDIR)/usr/sbin/schema/
$(INSTALL) -D $(TOPDIR)/models/yang/*.yang $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/config/transformer/models_list $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/models/yang/common/*.yang $(DESTDIR)/usr/models/yang/
$(INSTALL) -D $(TOPDIR)/models/yang/annotations/*.yang $(DESTDIR)/usr/models/yang/
cp -rf $(TOPDIR)/build/rest_server/dist/ui/ $(DESTDIR)/rest_ui/
cp -rf $(TOPDIR)/build/cli $(DESTDIR)/usr/sbin/
cp -rf $(TOPDIR)/build/swagger_client_py/ $(DESTDIR)/usr/sbin/lib/
cp -rf $(TOPDIR)/src/cvl/conf/cvl_cfg.json $(DESTDIR)/usr/sbin/cvl_cfg.json

ifeq ($(SONIC_COVERAGE_ON),y)
echo "" > $(DESTDIR)/usr/sbin/.test
endif

$(addprefix $(DEST)/, $(MAIN_TARGET)): $(DEST)/% :
mv $* $(DEST)/

clean: rest-clean
$(MAKE) -C src/cvl clean
$(MAKE) -C src/translib clean
$(MAKE) -C src/cvl/schema clean
$(MAKE) -C src/cvl cleanall
rm -rf build/*
rm -rf debian/.debhelper
rm -rf $(BUILD_GOPATH)/src/github.com/openconfig/goyang/annotate.go

cleanall:
$(MAKE) -C src/cvl cleanall
rm -rf build/*
90 changes: 90 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
## SONiC Management Framework Repo

### Build Instruction
Please note that the build instruction in this guide has only been tested on Ubuntu 16.04.
#### Pre-rerequisit
##### User permissions:
`sudo usermod -aG sudo $USER`
`sudo usermod -aG docker $USER`

##### Packages to be installed:
`sudo apt-get install git docker`

#### Steps to build and create an installer
1. git clone https://github.com/project-arlo/sonic-buildimage.git
2. cd sonic-buildimage/
3. sudo modprobe overlay
4. make init
5. make configure PLATFORM=broadcom
6. Run the prefetch python script to download all binaries (see below for the script).
7. To build mgmt-framework container:
`BLDENV=stretch make target/docker-sonic-mgmt-framework.gz`
8. To build Debian Stretch, if not already downloaded:
`BLDENV=stretch make stretch`
9. To build the ONIE installer:
`BLDENV=stretch make target/sonic-broadcom.bin`

#### Faster builds
In order to speed up the process of build, you can prefetch the latest debian files from Azure server, and just build what you need.

Here is a python script you could use to fetch latest prebuilt objects (deb, gz, ko, etc) from SONiC Jenkins cluster:

import os
import shutil
import urllib.request
from html.parser import HTMLParser

UPSTREAM_PREFIX = 'https://sonic-jenkins.westus2.cloudapp.azure.com/job/broadcom/job/buildimage-brcm-all/lastSuccessfulBuild/artifact/'

def get_all_bins(target_path, extension):
"""Get all files matching the given extension from the target path"""
print('Fetching %s*%s' % (target_path, extension))
os.makedirs(target_path, exist_ok=True)

req = urllib.request.urlopen(UPSTREAM_PREFIX + target_path)
data = req.read().decode()

class Downloader(HTMLParser):
"""Class to parse retrieved data, match against the given extension,
and download the matching files to the given target directory"""
def handle_starttag(self, tag, attrs):
"""Handle only <a> tags"""
if tag == 'a':
for attr, val in attrs:
if attr == 'href' and val.endswith(extension):
self.download_file(val)

@staticmethod
def download_file(path):
filename = os.path.join(target_path, path)
freq = urllib.request.urlopen(UPSTREAM_PREFIX + target_path + path)

print('\t%s' % path)
with open(filename, 'wb') as fp:
shutil.copyfileobj(freq, fp)


parser = Downloader()
parser.feed(data)
print()

get_all_bins('target/debs/stretch/', '.deb')
get_all_bins('target/files/stretch/', '.ko')
get_all_bins('target/python-debs/', '.deb')
get_all_bins('target/python-wheels/', '.whl')
get_all_bins('target/', '.gz')



##### Incremental builds
Just clean up the deb's/gz that require re-build, and build again. Here is an exmple:

##### To build deb file for sonic-mgmt-framework

BLDENV=stretch make target/debs/stretch/sonic-mgmt-framework_1.0-01_amd64.deb-clean
BLDENV=stretch make target/debs/stretch/sonic-mgmt-framework_1.0-01_amd64.deb

##### To build sonic-mgmt-framework docker alone

BLDENV=stretch make target/docker-sonic-mgmt-framework.gz-clean
BLDENV=stretch make target/docker-sonic-mgmt-framework.gz
3 changes: 3 additions & 0 deletions config/transformer/models_list
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#List yang models transformer need to load
openconfig-acl.yang
openconfig-acl-annot.yang
5 changes: 5 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
sonic-mgmt-framework (1.0-01) UNRELEASED; urgency=low

* Initial release.

-- Prabhu Sreenivasan <[email protected]> Tue, 18 Jun 2019 00:25:19 +0000
1 change: 1 addition & 0 deletions debian/compat
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
9
19 changes: 19 additions & 0 deletions debian/control
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Source: sonic-mgmt-framework
Maintainer: Prabhu Sreenivasan <[email protected]>
Build-Depends: debhelper (>= 8.0.0),
dh-systemd
Standards-Version: 3.9.3
Section: net

Package: sonic-mgmt-framework
Priority: extra
Architecture: amd64
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: SONiC Management Framework

Package: sonic-mgmt-framework-dbg
Priority: extra
Architecture: amd64
Section: debug
Depends: sonic-mgmt-framework (=${binary:Version})
Description: debugging symbols for SONiC Management Framework
7 changes: 7 additions & 0 deletions debian/rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/make -f
%:
dh $@ --with systemd


override_dh_shlibdeps:
dh_shlibdeps --dpkg-shlibdeps-params=--ignore-missing-info -l$(shell pwd)/build/cli/target/.libs/:$(shell pwd)/build/cli/.libs/
Loading

0 comments on commit 8b199a9

Please sign in to comment.