Here’s an outline of what I’m assuming about you and what you already know, as well as what software you’ll need ready and installed on your computer.
I’ve tried to write this book with beginners in mind, but if you’re new to programming, I’m assuming that you’ve already learned the basics of Python. So if you haven’t already, do run through a Python beginner’s tutorial or get an introductory book like 'Dive Into Python' or 'Learn Python the Hard Way', or, just for fun, 'Invent Your Own Computer Games with Python', all of which are excellent introductions.
If you’re an experienced programmer but new to Python, you should get along just fine. Python is joyously simple to understand.
I’m using 'Python 3' for this book. When I wrote it in 2013–14, Python 3 had been around for several years, and the world was just about on the tipping point at which it was the preferred choice. You should be able to follow this book on Mac, Windows, or Linux. Detailed installation instructions for each OS follow.
|
Tip
|
This book was tested against Python 3.6. If you’re on an earlier version, you will find minor differences (the f-string syntax for example), so you’re best off upgrading if you can. |
I wouldn’t recommend trying to use Python 2, as the differences are more substantial. You’ll still be able to carry across all the lessons you learn in this book if your next project happens to be in Python 2. But spending time figuring out whether the reason your program output looks different from mine is because of Python 2, or because you made an actual mistake, won’t be time spent productively.
If you are thinking of using PythonAnywhere (the PaaS startup I work for), rather than a locally installed Python, you should go and take a quick look at [appendix1] before you get started.
In any case, I expect you to have access to Python, to know how to launch it from a command line, and to know how to edit a Python file and run it. Again, have a look at the three books I recommended previously if you’re in any doubt.
|
Note
|
If you already have Python 2 installed, and you’re worried that installing Python 3 will break it in some way, don’t! Python 3 and 2 can coexist peacefully on the same system, particularly if you’re using a virtualenv, which we will be. |
I’m also assuming you have a basic grasp of how the web works—what HTML is, what a POST request is, etc. If you’re not sure about those, you’ll need to find a basic HTML tutorial; there are a few at http://www.webplatform.org/. If you can figure out how to create an HTML page on your PC and look at it in your browser, and understand what a form is and how it might work, then you’re probably OK.
The book uses the Django framework, which is (probably) the most well-established web framework in the Python world. I’ve written the book assuming that the reader has no prior knowledge of Django, but if you’re new to Python 'and' new to web development 'and' new to testing, you may occasionally find that there’s just one too many topics and sets of concepts to try and take on board. If that’s the case, I recommend taking a break from the book, and taking a look at a Django tutorial. DjangoGirls is the best, most beginner-friendly tutorial I know of. The official tutorial is also excellent for more experienced programmers.
See below for installation instructions for Django.
There’s a little bit of JavaScript in the second half of the book. If you don’t know JavaScript, don’t worry about it until then, and if you find yourself a little confused, I’ll recommend a couple of guides at that point.
If you’ve come from the world of Java or .NET, you may be keen to use an IDE for your Python coding. They have all sorts of useful tools, including VCS integration, and there are some excellent ones out there for Python. I used one myself when I was starting out, and I found it very useful for my first couple of projects.
Can I suggest (and it’s only a suggestion) that you 'don’t' use an IDE, at least for the duration of this tutorial? IDEs are much less necessary in the Python world, and I’ve written this whole book with the assumption that you’re just using a basic text editor and a command line. Sometimes, that’s all you have—when you’re working on a server for example—so it’s always worth learning how to use the basic tools first and understanding how they work. It’ll be something you always have, even if you decide to go back to your IDE and all its helpful tools, after you’ve finished this book.
Aside from Python, you’ll need:
- The Firefox web browser
-
Selenium can actually drive any of the major browsers, but Firefox is the best to use as an example because it’s reliably cross-platform and, as a bonus, is less sold out to corporate interests.
- The Git version control system
-
This is available for any platform, at http://git-scm.com/. On Windows, this comes with the Bash command-line which is needed for the book.
- A virtualenv with Python 3, Django 1.11 and Selenium 3 in it
-
Python’s virtualenv and pip tools now come bundled with Python 3.4+ (they didn’t always used to, this is a big hooray). Detailed instructions for preparing your virtualenv follow.
- Geckodriver
-
This is the driver that will let us remotely control Firefox via Selenium. I’ll point to a download link in the "Installing Firefox" section below.
Windows users can sometimes feel a little neglected in the open source world, since OS X and Linux are so prevalent, making it easy to forget there’s a world outside the Unix paradigm. Backslashes as directory separators? Drive letters? What? Still, it is absolutely possible to follow along with this book on Windows. Here are a few tips:
-
When you install Git for Windows, make sure you choose "Run Git and included Unix tools from the Windows command prompt". You’ll then get access to a program called "Git Bash". Use this as your main command prompt and you’ll get all the useful GNU command-line tools like
ls,touch, andgrep, plus forward-slash directory separators. -
Also in the Git installer, choose "Use Windows' default console", otherwise Python won’t work properly in the Git-Bash window.
-
When you install Python 3, unless you already have Python 2 and want to keep it as your default, tick the option that says "Add Python 3.6 to PATH" as in Add python to the system path from the installer, so that you can easily run Python from the command line.
|
Tip
|
The test for all this is that you should be able to go to a Git-Bash
command prompt and just run python or pip from any folder.
|
MacOS is a bit more sane than Windows, although getting pip installed was
still fairly challenging up until recently. Since the arrival of 3.4, things
are now quite straightforward:
-
Python 3.6 should install without a fuss from its downloadable installer. It will automatically install pip, too.
-
Git’s installer should also "just work".
Similarly to Windows, the test for all this is that you should be able to open
a terminal and just run git, python3, or pip from anywhere. If you run
into any trouble, the search terms "system path" and "command not found" should
provide good troubleshooting resources.
|
Tip
|
You might also want to check out Homebrew. It used to be the only reliable way of installing lots of Unixy tools (including Python 3) on a Mac.[1] Although the normal Python installer is now fine, you may find homebrew useful in future. It does require you to download all 1.1 GB of Xcode, but that also gives you a C compiler, which is a useful side effect. |
I’ll provide step-by-step instructions for Git, but it may be a good idea to
get a bit of configuration done now. For example, when you do your first
commit, by default 'vi' will pop up, at which point you may have no idea what
to do with it. Well, much as vi has two modes, you then have two choices. One
is to learn some minimal vi commands '(press the i key to go into insert mode,
type your text, press <Esc> to go back to normal mode, then write the file
and quit with :wq<Enter>)'. You’ll then have joined the great fraternity of
people who know this ancient, revered text editor.
Or you can point-blank refuse to be involved in such a ridiculous throwback to
the 1970s, and configure Git to use an editor of your choice. Quit vi using
<Esc> followed by :q!, then change your Git default editor. See the Git
documentation on
basic Git configuration.
Firefox is available as a download for Windows and OSX from https://www.mozilla.org/firefox/. On Linux, you probably already have it installed, but otherwise your package manager will have it.
Geckodriver is available from https://github.com/mozilla/geckodriver/releases. You need to download and extract it and put it somewhere on your system path.
-
for OSX or Linux, I recommend you put it in
~/.local/bin -
for Windows, put it in your Python "Scripts" folder
To test that you’ve got this working, open up a Bash console and you should be able to run:
geckodriver --version geckodriver 0.16.1 The source code of this program is available at https://github.com/mozilla/geckodriver. This program is subject to the terms of the Mozilla Public License 2.0. You can obtain a copy of the license at https://mozilla.org/MPL/2.0/.
If it doesn’t work, it may be that '~/.local/bin' isn’t on your PATH
(this would apply to some Mac and Linux systems). It’s a good idea to
have this folder on your path because it’s where Python will install
things when you use pip install --user. Here’s how to add it in your
'.bashrc':
echo 'PATH=~/.local/bin:$PATH' >> ~/.bashrc
Close your terminal and re-open it and see if the geckodriver --version
works now.
A Python virtualenv (short for virtual environment) is how you set up your environment for different Python projects. It allows you to use different packages, eg different versions of Django, and even different versions of Python, in each project. And because you’re not installing things system-wide, it means you don’t need root permissions.
Virtualenv has been included in Python since version 3.4, but I always recommend a helper tool called "virtualenvwrapper". Let’s install that first (it doesn’t matter which version of Python you install it for).
# on Windows pip install virtualenvwrapper # on MacOS / Linux pip install --user virtualenvwrapper # then make Bash load virtualenvwrapper automatically echo "source virtualenvwrapper.sh" >> ~/.bashrc source ~/.bashrc
|
Note
|
If Windows, virtualenvwrapper will only work inside the "Git-Bash"
shell, not from the normal command line.
|
Virtualenvwrapper keeps all your virtualenvs in one place, and provides convenient tools for activating and de-activating them.
Let’s create a virtualenv called "superlists"[2] that has Python 3 installed:
# on MacOS/Linux:
mkvirtualenv --python=python3.6 superlists
# on Windows
mkvirtualenv --python=py -3.6 -c"import sys; print(sys.executable)" superlists
# (a little hack to make sure we get a python 3.6 virtualenv)
Whenever you work on the book, you’ll want to make sure your virtualenv is
"active". You can usually tell because you’ll see (superlists) in brackets,
in your prompt. Something like this:
$
(superlists) $
Straight after you create your virtualenv, it should be active. You can
double-check by running which python:
(superlists) $ which python /home/harry/.virtualenvs/superlists/bin/python # (on Windows, it will be something like # /C/Users/IEUser/.virtualenvs/superlists/Scripts/python) (superlists) $ deactivate $ which python /usr/bin/python $ python --version Python 2.7.12 # for me, outside my virtualenv, "python" defaults to Python 2. $ workon superlists (superlists) $ which python /home/harry/.virtualenvs/superlists/bin/python (superlists) $ python --version Python 3.6.0
|
Tip
|
To activate your virtualenv, it’s workon superlists. To
check whether it’s active, look for the (superlists) $ in
your command prompt, or run which python.
|
We’ll install Django 1.11 and the latest Selenium, Selenium 3.
(superlists) $ pip install "django==1.11" "selenium>3" Collecting django==1.11 Using cached Django-1.11-py2.py3-none-any.whl Collecting selenium>3 Using cached selenium-3.4.1-py2.py3-none-any.whl Installing collected packages: django, selenium Successfully installed django-1.11.5 selenium-3.4.1
If you’re new to virtualenvs, or even if you’re not to be honest, at some point you’re 'guaranteed' to forget to activate it, and then you’ll be staring at an error message. Happens to me all the time. Here are some of the things to look out for:
ImportError: No module named selenium
Or:
ImportError: No module named django.core.management
As always, look out for that (superlists) in your command prompt, and a
quick workon superlists is probably what you need to get it working
again.
Here’s a couple more, for good measure:
bash: workon: command not found
This means you skipped a step earlier, and you haven’t added virtualenvwrapper
to your '.bashrc'. Go find the echo source virtualenvwrapper.sh commands
above and re-run them.
'workon' is not recognized as an internal or external command, operable program or batch file.
This means you’ve launched the default Windows command prompt, cmd, instead of Git-Bash. Close it and open the latter.
Happy coding!
|
Note
|
Did these instructions not work for you? Or have you got better ones? Get in touch: [email protected]! |
brew puts the Firefox binary in a strange location, and it confuses Selenium. You can work around it, but it’s simpler to just install Firefox in the normal way.