PlanoTse – A Must for Google Users

`

PRESS RELEASE

PlanoTse – A Must for Google Users

`

July 6, 2012
Plano, Texas

`

FOR IMMEDIATE RELEASE

`

PlanoTse, a personal tool for mining global data, is now available. Still in its infancy, this software can already do all of the following automatically. It generates multiple search strings, runs each per iteration, combines all search engine result pages, and filters documents as specified in web search scripting language. With this you can finally get your computer working as hard as you do.

One of the earliest uses of the software is in job search automation. That is why its first user’s guide is titled, PlanoTse Handbook for Job Search Automation. Try this for size, give PlanoTse your 3 job titles, 4 industries and 5 cities. It will generate 60 search strings and do all 60 web search iterations each night so you can catch new job opportunities in time. PlanoTse helps you leave no stone unturned on your search engine result pages.

The web search scripting language used in this app is called PlanoTseScript. It is simple yet powerful. Here is a foretaste: { must have Job Description, job description; must have Software Developer, Software Engineer; must avoid Hardware, Firmware; must avoid Instruction, Training; }. You can have as many such statements as you need to control your search precision.

As of this press release, the development of PlanoTse is basically a one-man show. Its creator is Tony, a Software Engineer who lives in Plano, Texas. The suffix Tse is hence an abbreviation for Tony – Software Engineer.

# # #

Web search while you sleep

This article tells you about my personal experience with the using of PlanoTse for web search automation. Specifically, it is for general job search and not tied to any particular job search engine.

Sure, the PlanoTse Handbook for Job Search Automation has all the guidance you need but it is so thick that it is hard to wrap your mind around all the concepts therein. This article should provide the full context for your first attempt at using PlanoTse for web search automation.

Text Me When Done

Yes, PlanoTse can automatically do the web search all night long while you sleep. And if you want to get up as soon as the web search automation is over, you can ask PlanoTse to text your phone at the end of its mission. Don’t forget to pump up the volume such that the text message, as shown below, can actually wake you up. The following image sums up one of the most beautiful features of PlanoTse as a personal tool for mining global data. Here is a picture of my iPhone getting a text from PlanoTse:

I’m a QTP Automation Consultant. That means I use QuickTest Professional (QTP) and program in VBScript. I want  PlanoTse to automatically search for the newest opportunities in QTP every night. In this way, I can always catch the best possible project opportunities for me and yet at the earliest possible time. This is how PlanoTse can help me to ensure that the time I spend in between contracts would be less and less.

Web Search Automation Wizard

I tell PlanoTse what to do by using a web search scripting language called PlanoTseScript. Since I do not remember so many different keywords in PlanoTseScript, I call for assistance from its Web Search Automation (WSA) wizard, a.k.a. automation wizard. The wizard is used before the start of automation. While automated web search is in progress, the automation wizard is not involved at all.

When you supply a search string for Google to find something, it comes back with several search engine results pages (SERP), each with up to ten document links. You click around until you decide to supply another search string. From the start of executing a search string until the time you try another search string is called a web search iteration. In PlanoTse, an automation iteration includes all activities related to a single search string.

To bring up the automation wizard, I select the menu [Automation >> Web Search]. Right on the first page of the wizard, I specify Factor Count to be 3 and set Factors to be: “major, minor, city”. This would make the next three pages to be Major Factor Page, Minor Factor Page and City Factor Page. I used two keywords for Major factor: “automated, automation”. I used two keywords for Minor factor: “QA, Test”. I do not worry too much about upper and lower cases as the search engine will automatically consider both for me. For the search factor City, I use 9 city names. These are of the cities along a freeway near home. From the 2 major, 2 minor and 9 city keywords, or phrases, the wizard will generate 36 different search strings, shown below, for 36 different search iterations in the same WSA session.

The above specification is not exactly what I want to do every night. I do not know that yet. This is only a start, to explore available possibilities. I will become more and more precise later, as explained further down in this article.

In PlanoTse, a run is a shorthand for a halfway decent HTTP response. A broken link, for example, is not counted as a run. You can declare a maximum number of runs per iteration as 20 in one of the following ways:

• Use the following PlanoTseScript instruction: { aw set maxruns 20; }. For more syntax, see this page.
• On page 1 of the automation wizard, set Maximum Runs to be 20.

I used 20 at first because I only want to have a foretaste and see if it is really necessary to use all 9 city names. After reviewing 20 responses from each of the 36 iterations, I will decide to keep all 9 city names, or not. If most the jobs I want are available in City1 and none are available in City2, I would not include City2 in my city list. This would save me from going through a lot of results from a set of useless automated search iterations for City2.

The group of all settings declared together by the automation wizard is called a configuration for web search automation, or simply, WSA config. I pressed NEXT again and again until the last page then I press GO to associate this config with the Go command on the main menu. To start the WSA session, I close the wizard then select the menu Go.

Web Search Scripting Language

A round of automation, also known as an automation iteration, is the duration between the start of the first iteration and the end of the last iteration. After the first round, I saw that I was getting many job listings. I hate that. I want to look at job offers, not job listings. To resolve this issue, I use a page filter consisting of instructions as listed below in PlanoTseScript syntax:

1. must avoid hardware, Hardware;
2. repeat maximum 4 more;
3. repeat maximum 4 detail;
4. must have Automation, automation;
5. must have Consultant, consultant, Profesional, professional, Analyst, analyst, Tester, tester;
6. must have QA, Test, test, testing;
7. must have QTP;
8. must have Job Description, Job description;
9. must have Minimum, Required, minimum, required; must have Compensation, Pay, Rate, Salary;
10. must have excellent, strong;
11. must have ability, skills;
12. must have process, plan;

What you see here is a foretaste of a web search scripting language. The above instructions make up a web search script. Web search scripting is much more precise than search string specification. The search engine rarely does exactly what you specify in the search string. In web search scripting, you must be case sensitive and very precise in defining your page filter. I can make this web search script longer and longer until most unwanted documents are filtered out. Only actual job descriptions would meet the exact criteria I spell out in a web search script with so many instructions.

For those who are new to PlanoTseScript, all must-avoid phrases must not be present in the document highlighted as an excellent search result. In contrast, only a single must-have phrase per filtering instruction must be present in the document.

The repeat-maximum filter is saying that, the document may not have more than so many times of a certain phrase or character string. This last feature is a way to fight against appearances of too many unwanted listing. Most listings have something repeated again and again on every row, for example, More or Detail, at end of each row briefing. The repeat-maximum should detect such a listing as forewarned by the user in their web search script.

WSA Configuration File

At this time, I realize that I’ve done a lot of work. Most of the work has been in figuring out what settings to declare for my automation. On the wizard menu, I select [File >> Save Config As...] to save my settings in a configuration file. For safekeeping, I copy the WSA config file to my thumb drive. When I’m at another computer, I can always download PlanoTse but I need to have this WSA config file to avoid starting from scratch in the development of my automation strategy.

 

 

Domain Exclusion Lists in Web Search Automation

One of the biggest concerns in automating your web search activities is how best to manage domain exclusion lists for your mission in web search.

Take a look at this domain exclusion list. It is a list of Google domain names not applicable to a number of web search activities. There are two types of domain names in this list:

• Proper domain name (without a slash)
• Extended domain name (with a slash)

PlanoTse can exclude both proper and extended domain names from the scope of your search mission. The procedure on this page in the PlanoTse handbook shows you manual steps for creating a new domain exclusion list.

PlanoTseScript Tutorial

After installing the software, PlanoTse 0.0.59 or later, you can, as a newbie to this software, take the Shortest Route in the handbook then walk through the tutorial via its Help menu. Select [Help » Tutorial » PlanoTseScript] then choose one of the available demonstrations. This page tells you about these tutorial sessions.

If you’re new to this and you want to have an insight to what you’re about to do, you may want to first read the topics as listed below, before viewing the following demonstrations.

• What is a data holder or a data item in PlanoTse?
• What is PlanoTseScript?
• What is a Go script?

Demo 1 – Associate a script with the Go command

In this tutorial, you will learn how to attach an existing script to the Go command on the main menu so that when you click Go the script will be executed.

Demo 2 – Work with source code in a script holder

In this tutorial,  you will learn how to open a script holder then access its source code.

Demo 3 – Web Search Automation & Scripting

In this tutorial, you will learn how to start a web search automation session using the PlanoTseScript web search scripting language.

How should I start with Market Research?

Are you new to market research? Good, so am I. Let’s see how we can get started with market research.

I’m the creator of PlanoTse software, a personal tool for mining global data, and I’m the inventor of PlanoTseScript, a web search scripting language.

I need to know if there is any product out there that is similar to PlanoTse and PlanoTseScript.

PlanoTse happens to be a tool that might help me. Let’s see what it can actually do for my real need.

I do the following steps to get started:

1. Select the menu command [View >> Code & Run]
2. Type in the input text box the search string, Steps in Market Research.
3. Press the Power Search button.
4. Press the Start button on the Popup Killer window.
5. Let the Search automation run overnight

The next morning, I select the menu [View >> Trace Log] and [Result >> Output Folder] for the result of my automated search.

The difference between doing this way as opposed to using Google directly is that I do not have to sit at the computer working with Google for hours.

Wow, many pages tell me what to do but I don’t understand how to apply the steps. They seem to be huge steps.

I’m going to do another automated search, using the phrase, “Example for Step 1 in Market Research”. I need to walk the dog anyway. Let’s see what PlanoTse can find out in 20 minutes.

When I return, I click on STOP NOW then let it finish doing the final reporting. Scroll to the bottom of the trace log for the table of prospective document links.

Cool, letting PlanoTse deal with Google for me is a great way to use Google even more while wasting much less time sitting in front of Google.

PlanoTse – Made in America

PlanoTse, a personal tool for mining the Web, helps you automate your web search activities. One of its earliest uses is in job search. What makes automation possible for PlanoTse is PlanoTseScript, a web search scripting language. It does all of the following automatically:

• Generate multiple search strings for multiple search iterations
• Combine all search engine result pages into one
• Filter document links based on a web search script

With such automation you can now make your computer work as hard at night as you do at work during the day. For example, in a job search scenario, from your input of 3 job titles, 4 industries and 5 cities, PlanoTse generates 60 search strings. Instead of manually running Google 60 times and tediously shifting through hundreds of result pages and thousands of document links, you can just let PlanoTse do all the work while you sleep. When all done, you only need to review relevant links on a single table. You no longer have to deal with broken links, crazy popups and unwanted ads. In short, PlanoTse is a must for Google users.

PlanoTse is a product made in America. To be more specific, it is made in Plano,  Texas. That’s why the prefix of the product name is Plano.

Actually, PlanoTse is not yet a bona fide product. It is still a prototype. Its first stable version is 0.0.58 and its first press release is not available until July 6th, 2012, exactly four months before our presidential election.

Do you live in or near Plano? If you live in Addison, Dallas, Carrollton, The Colony, Frisco, McKinney, Wylie, Sachse, Garland, Richardson, or anywhere within an hour of Plano, and you are interested in a killer app, you may want to contact the creator of PlanoTse while it is still a one-man show.

PlanoTseScript Web Search Scripting Language

PlanoTseScript is a scripting language for use in web search automation. To use PlanoTseScript you need to have PlanoTse, a desktop software tool for mining the global data off of the world-wide web.

PlanoTse newbies can use its wizards to generate various sample programs in PlanoTseScript. Once familiar with both PlanoTse and PlanoTseScript, you can tell your computer to do all of the following automatically through out the night:

1. Generates multiple search strings for a single run of web search automation
2. Use each search string in a separate web search iteration
3. Combine all search engine result pages into a single table
4. Filter and highlight documents as specified by the web search scripting language

Search String Generation

From your input of 3 job titles, 4 industries and 5 cities, PlanoTse generates 60 search strings.

Web Search Iteration

Each iteration is an execution of an independent PlanoTseScript program. A sample program is provided in another topic hereinafter. The last instruction in the program is the command, web search, to start the automation process.

In the earliest software versions, PlanoTse uses Google as the only search engine. At the start of each iteration, PlanoTse initially passes another search string to Google. This is done by the following steps:

• Generate an iteration-root URL to include the search string as a query to Google.
• Aim a browser at the iteration-root URL and wait for the first HTTP response from Google.

Merging Result Pages

From a single generated search string, Google generates many search engine result pages (SERP). Each link found on a SERP may be one of the following:

• A SERP link, which is a relevant link with an address to a SERP
• A document link, which is a relevant link with an address to a prospective document as a search result
• An irrelevant link, which is neither of the above

Within the same iteration, PlanoTse collect all SERP links and all document links from every SERP generated by Google. All broken links are thrown out. Pop-up windows are killed. Advertisements are ignored. At the end of the iteration, PlanoTse places all remaining SERP and document links in a single table of hyperlinks.

Filtering Search Results

The table of hyperlinks has three columns:

• Link number
• Hyperlink
• Parent link number

The following scheme is used to figure link and parent numbers:

• The link number for the iteration-root URL is 100. Its parent link number is zero.
• Number of the first link detected in the first HTTP response is 101. Its parent link is 100.
• Number of the second link detected in the first HTTP response is 102. Its parent link is also 100.
• Number of the last SERP link detected in the first HTTP response is, say, 199 as an example here.
• If the next link to be processed is 199 then the first link detected in its first HTTP response is 200 and its parent link number is 199.
• Etc.

In this way, the user can trace every link back to its parent SERP.

A Sample PlanoTseScript Program

A block of code in PlanoTseScript begins with an open curly bracket ({) and ends with a closing curly bracket (}). Each block of code can have zero, one or many statements. Each statement is ended with a semicolon (;). The last statement must not have the semicolon. Any text outside of code blocks are ignored by PlanoTseScript and thus considered as a comment. Below  is a sample program in PlanoTseScript that is used to control execution of web search automation (WSA). Line numbers are for discussion purposes and not part of the source code.

1. { clear responses; } — Delete all earlier HTTP responses stored on the hard drive.
2. { clear results; } — Delete all earlier web search results.
3. { clear websearch; } — Clear all in the web search temporary output directory.
4. { aw set maxruns 0; } — Set no limits and let each iteration reach the last possible SERP. The statement, aw set maxruns 5, means that the iteration is ended after receiving five relevant HTTP responses.
5. Etc.
6. For more possibilities, see the PlanoTseScript Language Reference in this handbook.
7. Etc.
8. { web search “Business Manager” + “Job Description” + -Retail + -Store; } — Ask Google to search using the provided search string: ”Business Manager” + “Job Description” + -Retail + -Store

Tell me more