Automation
Automation

Why Python is a Challenging Network Automation Language

by Juniper Employee on ‎04-12-2016 01:34 PM - edited on ‎08-23-2017 12:31 PM by Administrator Administrator (9,004 Views)

Sadly this isn't one of those ironic posts, sarcastically pointing out "failings" that are really strengths of the language. This is a true rant.

 

I'd like to say I am shining a light on the weaker areas of Python in an attempt to help make it better. But it's not. It's little more than a rant at a number of failures I see with Python. Hopefully my rant is good natured more than it is bad tempered.

 

My complaints really boil down to disagreeing with the statement "Python is a good language for novice programmers". And I'm not alone. All too often, Network Automation is going to be left to Network Engineers, not Software Developers. This means tools, resources, and applications that our industry will depend on will be written by earnest but inexperienced programmers. Instead of making our industry better, we're just going to leave a trail of mess for the next generation(s).

 

I am not trolling. I have literally lost sleep over this (and that is a badge of shame not honour). But I am genuinely interested in peoples views. Given the popularity of the language I must be missing something. So come on, convince me of the error of my ways.

 

It's Breaking Backwards Compatibility

Python has/is transitioning from 2.x to 3.x. This is a major change. As is common with such version changes, programs written in the old version may not work in the new version. This is true for Python. Here is a classic "Hello World!" program, print.py. It works fine in Python 2.x but doesn't in Python 3.x:

print "Hello World!"
> python --version
Python 2.7.10
> python print.py 
Hello World!
> python3 --version
Python 3.5.1
> python3 print.py 
File "print.py", line 1
print "Hello World!"
^
SyntaxError: Missing parentheses in call to 'print'

It's good that the error message is pretty helpful. But this does go to show the degree of incompatibility between the two versions.

 

The Python community should - rightly - be pushing for everyone to be using Python 3.x. The problem is there is a lot of code - a lot of libraries - that are not Python 3.x ready. For the networking industry, this is a particular problem as the majority of libraries you will use are written for Python 2.x.

 

OK, so what's the problem here (you might be thinking)? I'll just make all my Python apps use 2.x. Fine! That is until there is some feature of 3.x that you need, or some library you depend on becomes Python 3.x only. Then it is down to you to manage bits of the apps that are 2.x or 3.x dependent. And you have no control over how long you'll have to do that.

 

This really isn't Python's fault, and a lot of effort is going into minimising the headache of managing different versions of Python. It's just bad timing. Just as a new wave of (relatively) inexperienced programmers start using a language, it finds itself in the middle of a major release upgrade. But as unlucky as that may be it is something that has to be addressed - and quickly. A *lot* of code is going to be generated in Python soon. A lot of technical debt will be incurred until this is addressed.

 

It's Really Light on Syntax

One of Python's most advertised good qualities is that it is light on syntax - the letters, words, symbols used to define the structure of a program. A common example is "{ }" characters. However, Python isn't that light on syntax and in some cases is down right confusing.

Let's look at an example:

def foo():
    print "Hello World!"
foo

So what do you expect would be the output? You'd expect "Hello World!", no? Wrong! If you run this through Python you get nothing - no output at all! To get what you expect you need to change the last line to this:

foo()

It is the empty parentheses that tells Python to execute the function. But why? Why do I need empty parameters? If I have no arguments to give why do I need to pass an empty argument list (expressed as "()" at the end of the function call? Because in Python you can do this:

bar = foo
bar()

With this code, I'm passing a reference to the function to a variable. Then I'm calling the function by that variable. I'm sure there is a valid use case but I don't see the need to be so common that it forces you to add "()" to every functional call. Maybe it's just me, maybe everyone else is passing round functions all the time. Maybe this is what people really mean by Functional Programming. Maybe I'll stop the sarcasm now.

 

Python3 is an attempt to clear up some of the syntax inconsistencies that are in Python2. This is a very worthwhile effort. But they have decided to double down on "()". All function calls must include "()" or things won't work. Just see the breaking compatibility example above.

 

Objects and Types

Python is an Object Orientated programming language. Yet it is typed language, too. What this means is that Python will spit out an error if it thinks it's not obvious what you mean. Here's an example:

 

print(3)
print("3" + " is the magic number")
print(3 + " is the magic number")

Which outputs this:

> python typed.py
3
3 is the magic number
Traceback (most recent call last):
File "typed.py", line 3, in <module>
print(3 + " is the magic number")
TypeError: unsupported operand type(s) for +: 'int' and 'str'

 

So what's wrong here? The first line shows that python knows how to convert an `int` (the number 3) into a `str` to print, when it's by its own. `+` is used for string concatenation and works fine when both sides are a string.

The problem arises in the last line. The `int` type and the `str` type both implement the `+` operator, however they each do different things. But Python knows that one side of the operator is a string. Python knows that the final result is to print a string. Python knows how to convert an `int` into a string. Yet Python dumps a TypeError in your lap and leaves you to pick up the mess.

 

Thanks. For. That.

 

If only there was a principle that dealt with these annoyances and made things simple for the novice programmer.

 

It has List Comprehensions

Just what the heck does this mean!?

 

list = ["spam", "spam", "spam"]
[print(item for item in list)]

 

Pythonistas love list comprehensions. Understandable, as it's very pythonic. But this doesn't logically scan. I am referencing and using the `item` variable before I define it. How on earth do I know what to do with a variable before using it!? It's stuff like this that just makes my brain hurt...

 

It has Scary Syntax

Once you start writing objects, that whole "light syntax" really does just go out of the window. Want to define the Object constructor, you must def __init__():.

 

Python has a number of `__<something>__` methods. If you want to be helpful, you quickly find yourself defining a lot of them. The problem is these method names are not novice friendly. It feels like you're messing with the guts of the language, like you're into advance user territory.

 

And don't even get me started on the if __name__ == "__main__": idiom.

 

It has Annoyingly Repetitive Syntax

On purpose I omitted a critical piece when defining the Object constructor. What it really should look like is this:

 

class MyObject:
    def __init__(self):

 

So what is this `self`? Why, it's the object instance. i.e. if you created 3 MyObjects, the `self` would reference any one of those three.

 

OK, but why is that annoying? Because you have to add `self` to the argument of every instance method definition! Why? Who else will these instance methods refer to? It makes writing objects look like this:

 

class MyObject:
    def __init__(self):
        pass
    def method1(self):
        pass
    def method2(self):
        pass
    def methodN(self):
        print "kill me now"

 

Not annoying in the slightest...

 

White Space Matters, like a lot

Python is well known for using indentation to define a block of code. Once the block ends, the indent is shifted towards the left margin (usually by 4 spaces). Everyone knows that lines in a block must left align. But tabs and spaces can be treated equally by python but can look totally different depending on the text editor you use.

 

Here is the same code as viewed by two different text editors. Oh, and Python happily runs it, too.

Screen Shot 2016-04-12 at 12.35.55.png

indent.png

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

> python indent.py
string is in scope
string is in scope
string is in scope
string is in scope

Yes, I know, Pythonistas are zealots about using 4 spaces for all indentation. And all modern editors help enforce this rule. But seeing such examples as above is going to happen. At best it is just going to annoy someone while they fix it, at worse it will confuse someone and they will introduce a bug trying to fix it.

 

 

And seriously, white space should only be important for readability - not flow control. Well, for the possible exception of this programming language.

 

Aaaaaaaand breathe

That's about it. Well, not really. I could go on. But I think you get the message.

 

So, do tell, am I looking at this all wrong? Am I just indoctrinated into other languages and have become blind to Python's simplicity? Or do I have a point? Is Python really not a good language to put in the hands of non-full time programmers?

'
Comments
May 3, 2016
Super Contributor

Nice rant :)

 

And life gets more complicated (or more simple, depending on your perspective :) when you go to Python 3.x since it rejects the kind of mixed indentation that you have in your final example.  You can have either spaces or tabs but not both.

 

As a newbie to python and, despite having started programming with BASIC in around 1981 and then Pascal at University in 1987, I've never really been a talented programmer.  I write programs/scripts more out of necessity than a real passion :)  I've found Python to be pretty easy to learn enough to get by but doing things elegantly or efficiently is less obvious.

 

I actually like comprehensions when they bring a relatively simple for ... then construct down to a single line.  It's even useful if you have a for ... then if ... then ...  Your argument about using a variable before it's defined is true, but only if you parse either word by word or letter by letter.  If you take the whole line and parse it completely, then you are defining and using simultaneously. IMHO, used judiciously comprehensions can lead to more compact and still readable code.

May 3, 2016
Distinguished Expert

Hi

 

I think I see your point. Can you propose other language for beginner network automaters?

 

Some comments just IMHO:

 

- Tabs are evil and should never be used in programming, it is not Python's problem.

 

- Syntax requirements such as __double__ __underscores__ and self are a matter of habit and taste, and not really too important.

 

- What is important is dynamic typing of Python language, allowing to easily write simple scripts, but the same thing is a huge obstacle when using Python in large projects: troubleshooting becomes much harder compared to statically typed languages.

 

- Automation tasks do require programming skills, and no programming language by itself will make a programmer from a non-programmer (e.g. engineer), so again, the language is not too important.

 

May 4, 2016
Juniper Employee

Hi Guy,

 

Thanks for the comments.

 

I know that a lot of people get on well with Python - otherwise it wouldn't have such wide adoption!

 

The quicker Python (and by that I mean all the 3rd party libs) move to Pyhton3 the better. Just doing that doesn't address all my concerns but it would remove a massive hurdle.

 

I just hope we don't repeat the mistakes of previous automation attmepts. I fear Python makes that all to easy to do. Hopefully I'm wrong. 

May 4, 2016
Juniper Employee

Hi pk,

 

Thanks for the comments.

 

Yes tabs v spaces is an old "problem". But it is an accute one for Python. What I really object to is using indentation - alone - to signify block scope. I am all for syntactic breivity (I've done some XLST coding where it's almost impossible to see anything but syntax) but I think Python - in this regard - gets the balance of clarity of reading over clarity of comphention wrong.

 

As for 'self' and __double__underscores__ - it's not a preference thing. First of all you must define __init__() for your class to do anything useful. Second - tell it to the PEP8'ers. It's absolutely right that a consistent style be recommended. But you will likely get short change when asking for help if you don't go someway towards conforming to PEP8.

 

Dynamic typing is a problem but one that can be overcome. Rather than depend on the type of object, just make sure the object accepts the message you send it. You might call that Duck Typing. Whatever you call it I think it's a valuable design pettern in OO languages.

 

My current preferred language is Ruby. For me it's patterns fit very closely with the way I think. Is it a good beginner language? That's up for debate. It's geared towards developer happiness and I think that's no bad thing - and is rather good at achieving it.

May 4, 2016
Distinguished Expert

 Hi David,

 

Thanks for your comments, I agree Ruby is a decent altrernative. However it is also sometimes breaking backwards compatibility, doesn't it?

 

Speaking about preference, what I actually meant was if one hates underscores and explicit self so much, then change of the language should really be considered, but many people just get used to it.