How to build a simple automation project with Selenium and Python
Python is one of the most popular programming languages due to its flexibility, it can be used in various industries (research, web standalone games, and others), has a diverse library, and counts with great support from the community. On top of all that it has a very human-readable code structure that is easy to get the hang of, making it beginner friendly for anyone that wants to get into coding. On that note, I’m not a Python programmer but I picked it for this project because of all the reasons that I just listed and it was a very pleasant experience. The project at hand uses selenium to fill out a form without human intervention automatically. Here is what I used to build it and how I approached it, and I hope this article will help you in some way.
Let’s start with some basics, for the automation part I used Selenium. Selenium is a free (open source) automated testing framework used to validate web applications across different browsers and platforms. You can use multiple programming languages like Java, C#, Python, etc to create Selenium Test Scripts. It is widely used in the industry to create automated end-to-end tests for applications but can also be used for anything related to automating processes within a browser.
For the programming language, I chose Python. Since I have worked with several other languages and frameworks, I wanted to see if the organization and the flexibility of creating a project could be replicated in Python. I took as a base the Javascript frameworks and how all the dependencies are organized and isolated using the package.json file, it turns out that in Python there are two options to organize your dependencies: pipfile and requirements.txt. It doesn’t matter which one you choose the file will contain all the packages that are used for the project and their version.
Between the two options, I chose the pipfile just because it sounded more serious than a .txt file. To use the pipfile you have to install a global package that is called pipenv, basically, this package creates an independent runtime environment for your project, where the packages will be installed just for that project (not globally) and will handle everything like any other package manager for other frameworks. So for installing your dependencies for the project, you just run pipenv install and that will do all the dependency installation. To install new packages into the project just run pipenv install <package_name> and it will be added to the pipfile. To check on the packages installed in the project run pipenv run pip list and it will print a list of all the dependencies. Now to run the virtual dedicated environment console just run pipenv shell and the prompt for the console will change and that is how you know that you are in an isolated environment. But if you don’t want to use the virtual environment just use pipenv run command to run it once in the dedicated environment. Using this will give several advantages for your development process, you get isolation of projects, so all the projects will have their own dependencies, reproducibility, so all the people on your team has the exact same setup of dependencies and their version, and finally, you have portability, since all the dependencies are local to the project you can move to another computer and install the packages and be good to go to continue working.
At this point we have the runtime all figured out, so let’s try to bring even more functionality from the Javascript frameworks into our Python. There is a way to use a .env file on Python and for that, you have to add the python-dotenv package to your project, with this you can import the load_dotenv module into your code and use the directives os.environ.get to get the value of any environment variable that you have in your .env file.
Now let’s move on to the requirements to have Selenium running on your project. Selenium not only requires the package on your pipfile but also you need to install the driver for the browser that you want to use for your automation project. That driver has to be installed in the machine you are working on (it is not a package for your project) and there has to be one driver for each browser you are planning to use. Without this, you will not be able to run the code since the first configuration that you make with Selenium is the browser.
After you installed the driver you can start doing stuff with your browser like going to a page using the get method and start looking into the HTML to get buttons or text fields. Here are the methods that I used to build the simple form filler.
- Load_dotenv: Load the data stored in the .env file and makes it available through os.environ.get(variable_name).
- webdriver.Chrome(): The webdriver method is from Selenium and whit this method we are selecting to use the Chrome browser. In reality, you will use the Chromium browser because even though Chrome is based on Chromium, the driver I downloaded was the actual Chromium driver so that is what the application used. When you initialize the browser a window will open and stay on stand-by to any instructions coming from the script. I stored the result in a variable called chrome_browser.
- chrome_browser.get(site_url): This makes a get request to the specified URL through the browser window controlled by Selenium.
- chrome_browser.find_element(By.ID, field_id): This method finds an element based on its html id with the specified id, there are other methods of looking for elements under the By module but the most effective one is by id.
- selected_element.click(): This method simulates a click on the elements, I use this to click buttons, go into text fields, or open dropdowns.
- selected_element.send_keys(key_combination): With this method, you can simulate the stroke of a key, you can send a string and it will fill the whole field with that text, or you can do it letter by letter as if it was you that were hitting the keys, I also use it to send an enter key and arrows to select stuff from dropdowns, to use those keys you need the key module from selenium.
Finally, I have been having some issues with my script where it closes the browser window when the script finishes even though I don’t explicitly tell it to shut. To fix this problem I just use the input command on Python which waits for the user to input something and then continues, this way the script doesn’t finish until the user hit enter on the console.
That is all I wanted to share with you in this article. Thank you for reading it all the way down, I hope it has helped you to understand a little bit more about Selenium and gets you into the automation of tasks using browsers. If you liked the article give it a clap and check out my other articles on different software development topics.
