Blogs

Scripting How-To: share-data

By Erdem posted 08-14-2015 15:36

  
Overview

Share configuration data over a number of devices. This applies to SLAX version 1.0 and higher.

Description

This script maintains common config groups among a set of routers. Each group to be shared contains a list of targets to which the contents of the group a copied. The contents of this group on remote machines are completely replaced by the incoming data.

Source Code
 
GitHub Links

The source code below is also available from GitHub at the following locations:

Example Configuration
1 system {
2     scripts {
3         op {
4             file share-data.slax {
5                 description "Share configuration data over a number of devices";
6             }
7         }
8     }
9 }
Example Input
01 phil-fake {
02     apply-macro share-data {
03         10.5.18.2;
04         10.5.22.2;
05         10.5.14.2;
06         10.5.14.1;
07     }
08     system {
09         location postal-code 10101;
10     }
11 }
12  
13 ---
14  
15 phil@dent> op share-data group phil-fake target 10.5.18.2
16 The following group(s) are not applied locally:
17     phil-fake
18 Apply locally now? [yes/no]: yes
19     successfully committed locally
20 sharing group 'phil-fake'
21     connecting to 10.5.18.2
22     connected; transfering group phil-fake
23     successfully committed on 10.5.18.2
Example Output
001 Title: Propagation of configuration information across multiple Juniper switches 

002 Author: Kevin Barker, Independent Technology Group (kevinjbarker@indeptec.com) 

003   

004 The primary script share-data.slax allows for the propagation of configuration information from one Junos device to another. This script takes advantage of the Junos group function to store information and allow that information to be pushed from one Juniper switch to another. 

005   

006 How to setup: 

007 ------------ 

008 1-Install the scripts on the unit(s) that you will use as the primary or master configuration switches 

009     a.  Copy share-data.slax to the /var/db/scripts/op directory 

010     b.  Copy lib-util.slax to the /var/db/scripts/import directory 

011 2-Implement the script in the configuration on the boxes it was installed on 

012     a.  Issue the following configuration level commands: 

013         i.  user@host# set system scripts op file share-data.slax 

014         ii. user@host# commit 

015 3-Configure the data to be propagated 

016     a.  Define the group(s) 

017     b.  Define the IP addresses for the switches to be updated 

018     c.  Define the data 

019 4-Execute the script as desired on the master switch 

020     a.  user@host> run op share-data 

021   

022 How it works: 

023 ------------ 

024 To implement you will build a group -  this is a Junos construct that assists in the management of configuration data. The group will contain the information to be pushed to both local switch (optional)and the remote switches. It will also contain the information needed to access the switches. 

025   

026 The script will read the group information, establish an ssh session to each remote switch and push out the information. It will create a matching group on each remote switch and will update that group every time the script is executed. 

027   

028 It will also create an entry to apply the group information on the switch. This entry will be in the form of an apply-groups command. 

029   

030 How it looks: 

031 ------------ 

032 Following is an example: 

033   

034 Master switch setup: 

035         user@host# set groups vlan-distro apply-macro share-data 192.168.3.249 

036         user@host# set groups vlan-distro apply-macro share-data 192.168.3.250 

037         user@host# set groups vlan-distro vlans vlan-10 vlan-id 10 

038     user@host# set groups vlan-distro vlans vlan-20 vlan-id 20 

039     user@host# set groups vlan-distro vlans vlan-30 vlan-id 30 

040 Master switch execution: 

041         user@host> run op share-data 

042         The following group(s) are not applied locally: 

043           vlan-distro 

044         Apply locally now? [yes/no]: yes 

045         results = { 

046          commit-configuration; 

047         } 

048         sharing group 'vlan-distro' 

049          target: 192.168.3.249 

050           connecting to 192.168.3.249 

051         root@192.168.3.249's password: 

052           connected; transfering group vlan-distro 

053         results = { 

054          commit-configuration; 

055         } 

056          target: 192.168.3.250 

057           connecting to 192.168.3.250 

058         root@192.168.3.250's password: 

059           connected; transfering group vlan-distro 

060         results = { 

061          commit-configuration; 

062         } 

063   

064 This shows the execution of the script. The first six lines show the application of the script to the local switch as the local switch IP was not defined in the group setup. The remaining lines show the application of the script to the two remote switches (IP 249 & 250) 

065   

066 Remote Switch Configuration (After Push): 

067     user@host# show 

068         ## Last changed: 2011-09-21 19:41:04 PDT 

069         version 11.1R2.3; 

070         /* 

071          * $Id$ 

072          * 

073          * ex4200-defaults.conf  - Default configurations for EX4200 

074          * 

075          * Copyright (c) 2010, Juniper Networks, Inc. 

076          * All rights reserved. 

077          */ 

078         groups { 

079             vlan-distro { 

080                 apply-macro share-data { 

081                     192.168.3.249; 

082                     192.168.3.250; 

083                 } 

084                 vlans { 

085                     vlan-10 { 

086                         vlan-id 10; 

087                     } 

088                     vlan-20 { 

089                         vlan-id 20; 

090                     } 

091                     vlan-30 { 

092                         vlan-id 30; 

093                     } 

094                 } 

095             } 

096         } 

097         apply-groups vlan-distro; 

098   

099     user@host# show vlans | display inheritance | except # 

100         vlan-10 { 

101             vlan-id 10; 

102         } 

103         vlan-20 { 

104             vlan-id 20; 

105         } 

106         vlan-30 { 

107             vlan-id 30; 

108         } 

109   

110 Caveats: 

111 Group information is inherited: If the information contained in the group is already configured on the switch it will not be updated. The existing configuration will remain in place. In turn if that information is then deleted from the group it will NOT be deleted from the base configuration. 

112   

113 Group information is displayed at the group level: By default group information is not displayed in the body of the configuration. To see group information use the following commands: 

114     user@host# show system location | display inheritance | except # 

115   

116 Deleting the group on the master switch does not remove it from the remote switches: If the decision ismade to stop using the script for data propagation please note that the remote switches will have to be edited to have the groups removed from them. 
SLAX Script Contents
 
001 version 1.0; 

002   

003 ns junos = "http://xml.juniper.net/junos/*/junos"; 

004 ns xnm = "http://xml.juniper.net/xnm/1.1/xnm"; 

005 ns jcs = "http://xml.juniper.net/junos/commit-scripts/1.0"; 

006 ns ext = "http://xmlsoft.org/XSLT/namespace"; 

007 ns exsl = "http://exslt.org/math"; 

008 ns func extension = "http://exslt.org/functions"; 

009 ns dyn extension = "http://exslt.org/dynamic"; 

010   

011 import "../import/junos.xsl"; 

012 import "../import/lib-util.slax"; 

013   

014 /* 

015  * This script maintains common config groups among a set of routers. 

016  * Each group to be shared contains a list of targets to which the 

017  * contents of the group a copied.  The contents of this group on 

018  * remote machines are completely replaced by the incoming data. 

019  * 

020  * The script detects which addresses are assigned to the local device 

021  * and will skip such addresses, so the local device can be listed among 

022  * the targets, allowing the config to be updated and shared from any 

023  * device on the list. 

024  * 

025  * The share-data script can be invoked in two ways.  With no 

026  * arguments, the script looks for any group containing an 

027  * apply-macro named "share-data".  The contents of the macro are 

028  * a list of targets to which the shared data should be copied. 

029  * 

030  * In addition, the share-data script can be invoked with a set of 

031  * command line arguments to detail the exact data (by config group 

032  * using the "group" parameter) and targets to which that data should 

033  * be copied (using the "target" parameter with a space-separated 

034  * list of targets). 

035  * 

036  * The "database" argument can tell the script to use the "candidate" 

037  * configuration database, which allows uncommitted data to be shared. 

038  * The default behavior is to use the committed configuration. 

039  * 

040  * "transform":  Each target listed in the "share-data" macro can 

041  * contain (as its value (in the name/value pairing)) the name of 

042  * a func:function that can be called with the contents of the config 

043  * group to be shared, and returns the data to be shipped to that 

044  * target.  This allows all manner of transformations to be implemented 

045  * inside his framework. 

046  */ 

047   

048 /* 

049  * Future work: It would be great to have a "generation" number that 

050  * is monotonically increasing, and can be used to avoid clobbering 

051  * data when the config has been changed and not shared. 

052  */ 

053   

054 var $arguments = { 

055     <argument> { 

056     <name> "group"; 

057     <description> "Name of config group to share"; 

058     } 

059     <argument> { 

060     <name> "database"; 

061     <description> "Database for configuration ('candidate' or 'committed')"; 

062     } 

063     <argument> { 

064     <name> "target"; 

065     <description> "Target for transfer (one or space-delimited list)"; 

066     } 

067 } 

068   

069 param $group; 

070 param $database = "candidate"; 

071 param $target; 

072 param $debug; 

073 param $macro = "share-data"; 

074 param $transform; 

075   

076 match / { 

077     /* 

078      * Standard op script fodder; make the top-level element all op 

079      * scripts need (op-script-results), and an element to keep 

080      * our output together (out).  This script really only makes 

081      * <output> data, so this isn't needed, but can still be useful 

082      * for dumping data.  Well, okay, I tend to use jcs:dump() for 

083      * dumping data, so the script will really not make any interesting 

084      * output, but we still follow the JUNOScript rules and emit these 

085      * tags to play by the rules. 

086      */ 

087     <op-script-results> { 

088     <out> { 

089         call main; 

090     } 

091     } 

092 } 

093   

094 /** 

095  * main: start by getting the configuration for all groups.  Then 

096  * call share to either the one group chosen by command line argument' 

097  * or for all groups with the share-data macro. 

098  */ 

099 template main { 

100     var $rpc = { 

101     <get-configuration database=$database> { 

102         <configuration> { 

103         <groups> { 

104             if ($group) { 

105             <name> $group; 

106             } 

107         } 

108         } 

109     } 

110     } 

111     var $config = jcs:invoke($rpc); 

112     if ($debug) { 

113     call jcs:dump($name = "config", $out = $config); 

114     } 

115   

116     if ($config/xnm:error) { 

117     <xsl:message terminate="yes"> { 

118         expr "error: " _ $config/xnm:error/message; 

119     } 

120     } 

121   

122     var $rpc2 = <get-configuration database=$database> { 

123     <configuration> { 

124         <apply-groups>; 

125     } 

126     } 

127   

128     var $apply = jcs:invoke($rpc2); 

129     if ($debug) { 

130     call jcs:dump($name = "apply", $out = $apply); 

131     } 

132   

133     /* 

134      * Check to see if we need to add this group locally 

135      */ 

136   

137     if ($group) { 

138     call check-apply($apply, $names = $config/groups); 

139   

140     if ($config/groups/name) { 

141         call share($config = $config/groups, $group); 

142     } else { 

143         expr jcs:output("no configuration to transfer"); 

144     } 

145     } else { 

146     var $names = $config/groups[apply-macro[name == $macro] 

147                     && not(@inactive)]; 

148     if ($names) { 

149         call check-apply($apply, $names); 

150   

151         for-each ($config/groups[apply-macro[name == $macro]]) { 

152         call share($config = ., $group = name); 

153         } 

154     } else { 

155         expr jcs:output("no share-data configuration groups found"); 

156     } 

157     } 

158 } 

159   

160 /* 

161  * Determine whether the group has been applied locally 

162  */ 

163 template check-apply ($apply, $names) 

164 { 

165     var $missing := { 

166     for-each ($names/name) { 

167         var $this = .; 

168         if (not(../@inactive) && not($apply/apply-groups[. == $this])) { 

169         <missing> $this; 

170         } 

171     } 

172     } 

173   

174     if (count($missing/node()) > 0) { 

175     expr jcs:output("The following group(s) are not applied locally:"); 

176     for-each ($missing) { 

177         expr jcs:output("    ", .); 

178     } 

179     var $prompt = "Apply locally now?"; 

180     var $response := { call promptyesno($prompt); } 

181     if ($response == "yes") { 

182         var $configuration := <configuration> { 

183         for-each ($missing) { 

184             <apply-groups> .; 

185         } 

186         } 

187   

188         if ($debug) { 

189         call jcs:dump($name = "new", $out = $configuration); 

190         } 

191   

192         var $conn = jcs:open(); 

193         if ($conn) { 

194   

195         var $results := { 

196             call jcs:load-configuration($connection = $conn, 

197                         $configuration, 

198                         $action = "merge"); 

199         } 

200   

201         if ($results/node()) { 

202             call jcs:dump($name = "results", $out = $results); 

203         } else { 

204             expr jcs:output("    successfully committed locally"); 

205         } 

206         expr jcs:close($conn); 

207         } 

208     } 

209     } 

210 } 

211   

212 /* 

213  * Share the group's config.  Calls transfer() to do the real work.  If 

214  * a specific set of targets was given on the command line, use those. 

215  * Otherwise use the targets given in the macro, avoiding those which 

216  * belong to the current device. 

217  * 

218  * The logic to decide if the address is our's (local or (direct and 

219  * next-hop is lo0.0) was confirmed by rpd-hackers, so hopefully it's 

220  * valid for all cases.  If not, please let me know. 

221  */ 

222 template share ($config, $group) 

223 { 

224     expr jcs:output("sharing group '", $group, "'"); 

225   

226     if ($debug > 1) { 

227     call jcs:dump($name = "config", $out = $config); 

228     } 

229   

230     if ($target) { 

231     var $targets = jcs:split(" ", $target); 

232     for-each ($targets) { 

233         call transfer($config, $target = ., $transform); 

234     } 

235   

236     } else { 

237     var $mac = $config/apply-macro[name == $macro]; 

238   

239     for-each ($mac/data/name) { 

240         var $target = .; 

241         expr jcs:output("  target: ", $target); 

242   

243         var $rpc = <get-route-information> { 

244         <destination> $target; 

245         } 

246   

247         var $res = jcs:invoke($rpc); 

248         var $rt = $res/route-table/rt/rt-entry[current-active]; 

249         var $proto = $rt/protocol-name; 

250   

251         if ($res/xnm:error) { 

252         expr jcs:output("    error: ", $res/xnm:error/message); 

253   

254         } else if ($proto == "Local" 

255                || ($proto == "Direct" && $rt/nh/via == "lo0.0")) { 

256         expr jcs:output("    skipping local address"); 

257         } else { 

258         call transfer($config, $target, $transform = ../value); 

259         } 

260     } 

261     } 

262 } 

263   

264 /* 

265  * Transfer (load replace and commit) the configuration on the 

266  * remote target. 

267  */ 

268 template transfer ($config, $target, $transform) 

269 { 

270     /* 

271      * If there is a transformation defined, we call it, passing in 

272      * the config and getting back the post-transform config. 

273      */ 

274     var $trans-config := { 

275     if ($transform) { 

276         var $tpath = $transform _ "($config)"; 

277         expr jcs:output("    transforming ", $target); 

278         var $new = dyn:evaluate($tpath); 

279         copy-of $new; 

280     } 

281     } 

282   

283     expr jcs:output("    connecting to ", $target); 

284     var $conn = jcs:open($target); 

285     if ($conn) { 

286     expr jcs:output("    connected; transfering group ", $config/name); 

287   

288     var $configuration = <configuration> { 

289         <groups replace="replace"> { 

290         if ($transform) { 

291             <name> $config/name; 

292             copy-of $trans-config; 

293         } else { 

294             copy-of $config/node(); 

295         } 

296         } 

297         <apply-groups> $config/name; 

298     } 

299   

300     var $results := { 

301         call jcs:load-configuration($connection = $conn, $configuration, 

302                     $action = "replace"); 

303     } 

304   

305     if ($results/node()) { 

306         call jcs:dump($name = "results", $out = $results); 

307     } else { 

308         expr jcs:output("    successfully committed on ", $target); 

309     } 

310   

311     expr jcs:close($conn); 

312     } else { 

313     expr jcs:output("    failed to connect"); 

314     } 

315 } 
XML Script Contents
 
01 <?xml version="1.0"?>
02 <script>
03   <title>share-data.slax</title>
04   <author>phil.shafer</author>
05   <synopsis>
06     Share configuration data over a number of devices
07   </synopsis>
08   <coe>op</coe>
09   <type>network</type>
10  
11   <description>
12 /*
13  * This script maintains common config groups among a set of routers.
14  * Each group to be shared contains a list of targets to which the
15  * contents of the group a copied.  The contents of this group on
16  * remote machines are completely replaced by the incoming data.
17  *
18  * The script detects which addresses are assigned to the local device
19  * and will skip such addresses, so the local device can be listed among
20  * the targets, allowing the config to be updated and shared from any
21  * device on the list.
22  *
23  * The share-data script can be invoked in two ways.  With no
24  * arguments, the script looks for any group containing an
25  * apply-macro named "share-data".  The contents of the macro are
26  * a list of targets to which the shared data should be copied.
27  *
28  * In addition, the share-data script can be invoked with a set of
29  * command line arguments to detail the exact data (by config group
30  * using the "group" parameter) and targets to which that data should
31  * be copied (using the "target" parameter with a space-separated
32  * list of targets).
33  *
34  * The "database" argument can tell the script to use the "candidate"
35  * configuration database, which allows uncommitted data to be shared.
36  * The default behavior is to use the committed configuration.
37  *
38  * "transform":  Each target listed in the "share-data" macro can
39  * contain (as its value (in the name/value pairing)) the name of
40  * a func:function that can be called with the contents of the config
41  * group to be shared, and returns the data to be shipped to that
42  * target.  This allows all manner of transformations to be implemented
43  * inside his framework.
44  */
45  
46   </description>
47  
48   <keyword>share</keyword>
49   <keyword>data</keyword>
50   <keyword>groups</keyword>
51   <keyword>config</keyword>
52   <keyword>remote</keyword>
53   <keyword>rpc</keyword>
54   <example>
55     <title>Example</title>
56     <description>Simple test case</description>
57     <config>example-1.conf</config>
58     <output>how-to.output</output>
59     <input>example-1.input</input>
60   </example>
61  
62   <xhtml:script xmlns:xhtml="http://www.w3.org/1999/xhtml"
63                 src="../../../../../web/leaf.js"
64             type="text/javascript"/>
65 </script>

 


#How-To