Frequently Posted Questions  (FPQ)

"How do I search for Information"

Or

Or this should be entitled 
"BBQ Search Does Not Search The BBQ Forum Archives"


By: Ray Basso

This definitely, is a frequently posted question.  It's a question I have asked since the BBQ Forum was first started in 1995.  An enormous amount of information has been posted to the forum since then.  As of the writing of this FPO in December 2003 over 300,000 messages have been posted to the forum.  How do you find a specific message you are looking for is the question.  

My first attempt at saving the information posted to the forum was to manually select good messages and manually organize them it a place where they could be read.  You can see this first attempt at the following web page which is a part of the Kansas City Bar B Q Connection ( The Kansas City B Q Connection was the first web page on the Internet about barbecue and it was one of the first web pages I wrote.).   http://www.rbjb.com/archives/thebest.htm  This soon became impossible to keep up because of the time required to do it and the problems of earning a living for my family. As the messages kept coming into the forum I couldn't keep up with the work.  So I adopted another method of saving this valuable information.  I tried to manually archive threads from the forum http://www.rbjb.com/rbjb/send.htm this link takes your to the early archive efforts.

This manual archiving of the messages continued for a long time.  It was very time consuming and required about an hour of work for each archive.  Somewhere during 1997 to 1998 I wrote the BBQ Search web page.  It's intent was to organize links posted to the BBQ Forum and select good message that you could search on.  With this web page I moved from the UNIX type of web page to the "new" MS FrontPage technology.  Again this was all manually done by me and very time consuming.  The best method I had to do the searching was the FrontPage "search bot" because of money and time limitations.  The search feature was flawed but it was the best I could do and I did manage to get over a 1,000 message in it before I gave up on this search project.  I have left it in place because there are a lot of very good messages in BBQ Search.  The search function is not very good but the alternative is to wipe out those 1000 good posts.  The big problem with BBQ Search. is that the search feature, searches every page in the bbqsearch.com web page and returns some off the subject information.  IMPORTANT The actual Archive files of the BBQ Forum are not on the same server as the bbqsearch.com web page and bbqsearch can't search the archives.

In addition to this problem no matter how many times I tell people that BBQ Search does not search the archives of the BBQ Forum, people continue to complain about it because it does not search all of the archives.  I even have the following warning near the search button:

The search feature of BBQ Search does not search the archives of the BBQ Forum.  It is intended to supply basic information only.  It does a simple search of every page on this web page.  A great deal of valuable information can be found by properly using this search feature.

Maybe I should make it bigger but I think no matter how big I make this warning some people will still not read it.

Some where, about this time, I tried to store archived messages on a CD and created the BBQ CD.  It's a heck of a deal, it even has two great cookbooks on it, but even though we have sold some, it never was popular enough to solve the problem.  So I have given up trying to store archived messages on CD's.  It seems that even though this might be very easy to use, there doesn't seem to be enough support to cover the time and costs of such a project.

Karl Shafer, came alone about this time and helped me a great deal with the script that operates the BBQ Forum.  Karl set it up so it automatically archives the Forum and therefore saved me hundreds of hours of work.  Unfortunately, Karl passed a way unexpectedly and I was forced to learn more about the scripts.

Then, another new problem reared it ugly head.  The web space, that the archived messages took up on servers, became to much for web hosting companies using shared servers.  So I started storing files and information on other web pages I owned.  But, I soon used up all of that space.  The only answer was to buy a dedicated server which involved more expense.  To offset that expense I had to take on several thousand of dollars of new expense to generate advertising to try to pay for everything. 

Then in order to make finding recipes easier I created the recipe search feature.  Over 2002 and 2003 I have manually added about 400 recipes to this search feature.  It works fairly well because all the files are stored in a sub web and your search results returns only recipes posted to the forum.  But getting people to use the recipe search is tough because they would rather complain about how BBQ Search.com doesn't work.

-------------------

That's about enough for the history.  The present problem is The BBQ Forum has generated about 350,000 html documents scattered over several web pages and more than one server.   There are search engines out there that will search these 350,000 documents ( two gig of data) but they are very expensive.  In November of 2003 I contacted many search engines and checked out a lot of software.  The only ones that I could find to do a good job and handle future growth would cost between $15,000 and $25,000 per year.  There is not enough money to solve this problem at this time.

I have looked at many search options and the problem is the amount of data. Google has a Search Appliance that can do a good job and safely. The same can be said of Atomz and others. Please give them a call and tell them you want to be able to search on 2 gig of data and 300,000 files. Then you can also inform them that this will grow to over 500,000 files in the next couple of years. Then ask them how much it will cost and you will see why I have a reluctance to spend $10,000 to $25,000 per year. That is my reluctance. Forget it, I donít have anywhere near that much money.

Atomz will search 500 files free if you display their logo. This is will not do us any good. We have more than 300,000 files and the last quote I got from them was $15,000 per year. I have talked to Google about their web Search Appliance and the quote from them was more that Atomz. I am sure there are other solutions and every time this is brought up I have a lot of people who seem to know more than I do, offer advice. However, none of it has ever solved the problem of keeping intact and searching all the valuable information posted to this forum and what can be expected in the future.

If your or anyone else can find a low cost, safe way to search over 300,000 files please let me know by email.

_____________

So how do you search for information at this time.  Well, you can use the BBQ Search function.  But if you do please don't complain about it not working well.  I know it is not the best.  However, it does contain a lot of good information if your are patient.  

Please remember that BBQ Search does not search The BBQ Forum Archives.

------------------

Then you can read the archives of the BBQ Forum.  That's a lot of work, but if you want the knowledge then you should read them.  You can speed up your search of the information in the archives by searching each archive file.  You do this by opening up an archive file and then pressing Ctrl+the F key and then entering the item your looking for.  Please remember that this will only search the information on the page your looking at and not the messages on for that archive file.

-----------------

You can use the recipe search function.  The link to this is located in the heading of the BBQ Forum.  There are a lot of recipes in it and it works fine.

-----------------

Probably the best method is to use the Google Advanced search feature.  You can access this by going to http://www.google.com and clicking on the advance search link.  Once there you enter the search term you want to search on and then enter that you want to search rbjb.com.  This works well at this time.  However, as Google grows and the need for money increases you can expect this to feature to go away someday.

 

Ray Basso

The BBQ Forum