Sunday 4 December 2016
Automating repetitive tasks in the browser
Imagine your boss comes very anxious at work and ask you an unholy task: he has a spreadsheet with 600 lines and he needs you to fill a form on the web for each one of them. He expects you access this form and fill every field some 600 times.
Well, there's a very interesting Firefox plugin called iMacros that will make your task a lot easier. iMacro allows you to record and play Macros. That means you click the REC button, do what you got to do and then click the STOP button. You then save the macro under any name you want and you are a click away from getting Firefox to do it all over again by itself. Neat.
In Firefox, click Tools > Complements and search for iMacros (or use the above link). After installing it will be necessary to restart Firefox. Once restarted you will see an icon before the address bar (there is a green gear in the icon). Click this icon and a side panel will show.
Notice that the panel has three tabs: "Use", "Record", "Edit".
Let's make a test: click "Record". He will ask if you want to close the other tabs (if there are any other opened) and after that it will be recording. Surf the web a little, access Google, make a little search and than press "Stop". All that you have done is recorded, you may click "Save" button to name your Macro. If you don't, it will remain under the name "
#Current.iim" and will be override as soon as you record a new macro. If you name your macro you will be able to easily find it in the future if you need.
Now click the "Use" button and you will see your browser working by itself, going through all pages you went before and typing everything you typed.
Filling a form based on a CSV file
Let's do it. We will use a spreadsheet to fill automatically the fields of a web form.
Step 1 - Recording a macro
Access the form your boss asked you to fill. Set iMacro to record and fill every field. Save the macro with a familiar name.
Step 2 - Preparing the csv file
iMacro doesn't work with Excel or LibreOffice. You'll need to open the spreadsheet and save it in CSV format. It will be necessary to remove any merged cells the file may contain.
With LibreOffice, when saving your spreadsheet, you'll be asked what symbol to use as a column separator and what text delimiter to use. Column separator should be the comma , and text delimiter should be double quotes. Save the CSV file with a simple name (no spaces or complex chars).
iMacro uses only files saved at the folder iMacros/Datasources that should be located in your personal folder (in Linux, usually
/home/username/iMacros/Datasources)Step 3 - Editing the macro file
We need to edit the saved Macro, otherwise it will always fill the form with the same values.
Open the folder iMacros/Macros (/home/username/iMacros/Datasources) and look for the file with the name you gave you macro. If you named your macro "testing" it will be named "
testing.iim".
Open the file.
In the beginning of the file (after
TAB T=1, if it is there) you will need to identify the file that will be used as source of the data we want to upload. Let's set the following variables:- !DATASOURCE as the name of the file that contain the data (CSV file);
- !DATASOURCE_COLUMNS the number of columns the file contains (this number must be accurate);
- !LOOP the number of the line that contains the first line of data (if your file has 2 lines of header use 3, if your file has no header use 1)
Here is the result for our example:
TAB T=1
SET !DATASOURCE data.csv
SET !DATASOURCE_COLUMNS 3
SET !LOOP 1
SET !DATASOURCE_LINE {{!LOOP}}
Now let's work the macro loop (the rest of the document). Check out the code iMacro generated. You will notice that it refers to the fields you filled and the positions your mouse clicked. Carefully find the which lines in the Macro corresponds to each field of the web form. Identify also the number of the column where the information for each field is saved in the data file.
For each field you will find in the code something similar to "
CONTENT = Adam Smith", for instance, to write "Adam Smith" in the form field. Replace the value written in the recorded Macro (the value you typed in the web form) for the column identifier. The column identifier will have the following syntax: {{!COL1}}, where 1 should be replaced by the number of the column that has the data. Thus, for a data located at the first column, use {{!COL1}}. For data at the second column, use {{!COL2}}.
Now let's check our example:
URL GOTO=http:/testing-url.test
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/node/add/pericia ATTR=ID:edit-title CONTENT={{!COL1}}
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/node/add/pericia ATTR=ID:edit-field-tipo-pericia-0-nid-nid CONTENT={{!COL2}}
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:node-form ATTR=ID:edit-submitStep 4 - Running the macro
It's now all set. Now you need to go to your browser and click at the "Use" tab. There you will find the "Use (loop)" button. Notice that above it there is a field "Max". Fill the "max" field with the number of lines you want to fill. That is, if your file has 500 lines and you want to fill up to the line 499, type 499. Now, click the "Use (loop)" button and iMacro will execute it. It will fill the form one time for each line in the data.csv file up to the 499th line.
Step 5 - Learn more
I am not sure if I explained it enough, but I learned it reading the iMacro wiki. You'll get a lot of info there.
Here is the code I've recently used as an example. I've experienced some problems related to the CSV file. It really needs to be saved with commas as column separators and double quotes as text delimiters. I suggest using LibreOffice calc to export CSV or editin CSV directly. Lot's of people had trouble using Excel.
TAB T=1 SET !DATASOURCE lista_dos_produtos_a_carregar.csv SET !DATASOURCE_COLUMNS 3 SET !LOOP 1 SET !DATASOURCE_LINE {{!LOOP}} URL GOTO=http://examplesite.test TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/node/add/pericia ATTR=ID:edit-title CONTENT={{!COL1}} TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/node/add/pericia ATTR=ID:edit-field-tipo-pericia-0-nid-nid CONTENT={{!COL2}} TAG POS=1 TYPE=INPUT:SUBMIT FORM=ID:node-form ATTR=ID:edit-submitfrom: http://ndvo.blog.br/
