Screen scraping as a data retrieval app to maximize inventory control and profit margins

Data collection as Custom software development to Increase Productivity and Profit Margins:

Our firm provides web scraping services to variety of clients. Lately we developed a screen scraping application as web based solution for one of our clients that was completely revolutionized the way they managed their inventory. It enables them to gain a real advantage against their competition by working at the cutting edge of technology and custom software development can offer in their sector. It was developed as a dot net application and we used multi thread capability and also known as parallel processing to maximize speed and performance.

This new web scraping application allows the client to maximize their profit margins and not be under sold against any other competitors.

Find the same part with a complete different part numbers for the same part, even though initially the client did not even know they already have it in stock. (See details below).

Can find multiples of different related part numbers to the original part number when the client searched upon.

What does web scraping development can do for you?

The data retrieval app, also known as screen scraping will search in parallel for multiple suppliers and competitors at the same time. It helps them to find the best purchase price and offer a very competitive selling price. The information retrieved includes part properties, images of the part, description with additional part numbers, other related parts with their part numbers as well.

In general, the first process is similar to one of content scraping application by grabbing the raw data from multiple web sites as a starting point.

Does data collection help as a Decision Support System?

It is much then more than just a screen grab. While it retrieves the data, it performs as a decision-making process for the client in a much more sophisticated way than any regular content scraping software they used previously.

Just by getting the data from multiple suppliers, sorting and storing the substantial amounts of new data in their local database in a logical way creates a major knowledge base for the client. The client can take that data and apply it within other applications as they see fit.

The Data Retrieval application then connects different part numbers from different suppliers and presents possibilities that any human Google or Bing search engine, could not find on their own. There are different levels of how far we connect each P/N from one supplier to another, for now we only drill down to the fifth level, but the app has the capability to go further if and when is needed.

Getting the best price for the part!

There are also multiple factors of how to obtain the best price from multiple suppliers and currencies. Our software developers had to develop an algorithm just for that. Since each supplier presents its data in very different ways, the content scraping application draws the different information together and presents it in a logical way to the user.

In general, it is not just to grab data as a screen scrapping application from web sites. It is what you do afterwards with that data and how you utilize it, so it helps you to beat the competition and increase the profit margins and basically increased tremendously the client’s bottom line.

Below are some points that our client indicated of how that web scrapping application benefited them: (These are the actual points the client wrote)

Searching Part Number for more details within the web scraping app and become one stop shop to all customers:

When we are researching information details for a part where we have little or no information, if we have a part number, we can put the part number in the web scraping app and examine the results on 2 x fronts:

Firstly, to see what info of our suppliers and competitors have about the part, such as fitments or application information, specific part details such as: dimensions or weight, part properties etc. This information can be checked out and added directly to our records. Secondly, we may find other part numbers in the search results and / or the tree view which we can pursue in terms of adding those part numbers as direct cross references.

In addition, running further searches on those numbers in the content scraping app to find further part details, a supplier or competitor that may have related this other number.

But all this additional data relates to the same part and therefore those details can be added to our part record, increasing our depth of information about any particular part number.

Eventually we can be the place where all our customer (Our client's customers) can find all the info, they need rather than having to spend a great deal of time checking out a large number of suppliers and trying to consolidate the info to make sense of things.

in other words, it helps us to be the easiest place to buy from for our customers because we will have pretty much every bit of info they need right there.

Having said that, it makes it easy to any potential buyer to quickly purchase from our web site since we became a one stop shop for all of them.

Optimize the search via the data collection application for the specific supplier(s) or competitors:

The administrator can create different groups of suppliers and competitors and assign them to different users or group.

Certain users looking for specific parts and it just does not make sense to search within all suppliers or competitors for some part numbers that the client knows that they don’t carry the part.

The purpose is mainly to reduce noise, if you are searching for a part in a particular section, it is of no use to see results from a supplier or competitor that does not deal in that area, so any results found are just ‘noise’ in terms of that specific search.

In addition, some of our suppliers are known to monitor accounts and usage closely, they only want to see relevant searches on the site from each user account.

For example, in one role for say a straight tele-sales role, they are only concerned with if the part is available from our suppliers and are not interested in our competitors. But, if you take a sales or purchase manager, seeing the data from the competition is a crucial part of the information they need.

Check a part number and no results:

Search for a part number and the application returns ‘No Data available’.

This apparent ‘negative’ is actually a ‘positive’ result because it instantly tells us that our set of suppliers and main competitors do not hold any relevant info for this part number, therefore we do not need to spend any more time on this query at this point in time as it is very unlikely that we, or anybody else would be able to supply the part to the customer based on this number

Check a part number and multiple supplier / competitor results are displayed:

In this case we can instantly see who could supply us with the part and in some cases at what price we need to pay for it, which obviously means we can identify the best price very easily, but also looking at the results from our competitors gives us insight into what the current market price might be and therefore what profit margin maybe available for the part.

Check a part number and only 1 supplier found, but no competitor results are displayed:

This tells us we can supply the part, but it is not widely available in our market sector, and we may be able to gain a larger profit margin on the sale.

Check a part number and it leads on to many other part numbers:

This is a less obvious process and involves a little interpretation and skill from the user, but does happen a surprising number of times.

We are provided with a number, say “XYZ123” the search returns a result from one or more supplier’s, but they show a different part number as being the same part, say a supplier shows “XYZ123” and “ABC123” as being legitimate numbers for the part.

We then follow the audit trail for “ABC123” possibly running that reference in the Data Retrieval application and getting new results from a different supplier who recognizes “ABC123” but does not recognize “XYZ123”

BUT[!] this supplier shows their refs include yet another number “XXX789” as being the same part as “ABC123” which of course is further directly linked to “XYZ123” from the previous supplier.

This is NOT fantasy, this situation has occurred numerous times, which means we can make a sale of a part we didn’t even realize we had sitting right there in stock at the time of the initial inquiry!

The above process is under ongoing development with The Farber Consulting Group Inc. to expand and improve this capability to relate seemingly unrelated part numbers.

Based on the above description the screen scrapping software will create the part number relations to the user while display all parts via a tree view presentation as follows.

Here is the hierarchy below as described above. The part numbers shown below are made numbers.

In general, the client can search for multiple part numbers and while using their ms sql database as the internal knowledge base to find suppliers that can match any part number.

In addition, find the same part but with a completely different part numbers that were UN-known to the client and see the hierarchy as shown below.  Every part which is highlighted in the tree view provides the supplier’s info plus the part details including its properties and other related part numbers.

For more questions about our services please call us at: 732-536-4765. Or Contact Us via our web site. You may also check our database department for our Remote DBA solutions.

Technologies we use:

Dot Net, Alpha Five, Alpha AnyWhere, Java Script, MS SQL 2019 or lower, MySQL database, SSIS - SQL Server Integration Services, MS Access and Umbraco CMS - For web content management.

Additional Services that we Provide:

• Remote DBA services for MS SQL or MySQL
• Database design
• Database optimization
• Convert your Desktop Application into a web-based solution
• Screen scrapping services while using dot net
• Optimize your Stored Procedures
• Dot Net development
• Alpha AnyWhere Developers, also known as: Alpha Five Development
• Mobile application development Solutions
• Convert Visual Pro Apps to Web Application
• Maintain Visual FoxPro Apps
• Convert MS Access To Web