Thoughts on Web Bot

[Seth]

[I know this is an older issue mentioned in other posts but maybe we can soley address it here]

“The Web Bot Project, developed in the late 1990's, was created to assist in making stock market predictions. The technology uses a system of spiders to crawl the Internet and search for keywords, much like a search engine does.” (http://www.digg.com/odd_stuff/The_Web_Bot_Project_and_The_Year_2012)

"[Web Bots] scan a huge portion of the internet's public discussion boards and resolves the conversations going on in a huge pile of data going many gigabytes in length. This is then "distilled" using an artificial intelligence language (prolog) which throws out all comments which are past or present in tense. What's left is a smaller pile of forward-looking sentences. These are then refined and compared with previous scans of the web to get a pretty good indication of how things "might" go." (http://www.rense.com/general69/webbotrun806.htm)

It seems like a fascinating concept but must admit I am skeptical of any predictive device, even those based in technology. Those of you with technical prowess may be able to shed some light here. Can this thing even be legit theoretically?

 

[Rick Deckard]

It seems like a fascinating concept but must admit I am skeptical of any predictive device, even those based in technology. Those of you with technical prowess may be able to shed some light here. Can this thing even be legit theoretically?

Didn't some MIT dudes once crash the market in 1990's using their home-grown algorithms? If I recall, they based predictions purely on 'trend analysis' and made billions of dollars on the futures markets...


...except, the very fact that they were so successful at it in itself, affected the market and therefore brought it down.

I think it was lead by this guy - Myron Scholes - using his group Long-Term Capital Management - I saw a documentary, probably over ten years ago now, about they formulated their equations. Interesting stuff.

You should read up on Game Theory - pay attention to Nash's Equilibrium (it's why the Cold War started), that'll blow your mind.

 

[Seth]

Very interesting post Rick. Wikipedia has the tendancy to send me on deep quests for long stints due to its inherent vocabulary and linked subject matter on topics like this.

We could assume then that the constructs of a web bot calculate trends in speech and in popularity of topics based on internet dialogue. And can appear to predict future events or make prophecies based on the mood of a given populace. Therefore, a bot does nothing more than operate as a gauge or thermometer measuring the intensity or interest in given topics.

So, if a bot is a gauge, and the gauge measures some quantity the problem lies in the fact that we do not know if the quantity measured is an accurate depiction. In other words, the topics in discussion on the internet do not necessarily reflect an actual likelihood in real-time or real-life so may not be good tools to use as predictions.

I can see how trends in stocks and trading could benefit from a bot because the bartering of goods is at its core a trend-based system. How can we use the same technology to predict non-trend-based topics like the occurrences of future events?…It does not seem like the appropriate tool, does it?

 

[Rick Deckard]

I can see how trends in stocks and trading could benefit from a bot because the bartering of goods is at its core a trend-based system. How can we use the same technology to predict non-trend-based topics like the occurrences of future events?…It does not seem like the appropriate tool, does it?

Well, the intelligence agencies are 'trawling' all electronic communications (including voice) in an effort to predict (and possibly influence) future events.

Project Echelon

They obviously think that's there's some merit in that.

BTW, the American Intelligence Agencies have been accused of passing on confidential info to US business 'interests' that they've found in this way...

 

[Seth]

Well, the intelligence agencies are 'trawling' all electronic communications (including voice) in an effort to predict (and therefore prevent) future events.

Aren't these real-time captures of people making actual future commitments? Like: “I am going to commit this horrible act at this time.” Or “Meet me at the sand dune for a Jihad next week.” Not necessarily a set of algorithms computing future events based on the collective mind?

What I am asking is how does the text-based conversations predict events like 2012 as web bot forecasters like to claim?

 

[Rick Deckard]

What I am asking is how does the text-based conversations predict events like 2012 as web bot forecasters like to claim?

Okay, I missed the point.

My answer is that they probably can't predict anything like that unless they believe that humans are 'unconsciously receptive' to signals from the the future (and indeed the past) and this is reflected in everyday interactions - I can't rule that out of course.

Here's a related theory of mine - people with long hair are more emotionally 'tuned' to their fellow humans than those with short hair. Discuss.

 

[Ron Collins]

I understand how looking for trends can help predict stock. I can understand how it can help predict a bunch of other relatively short term things. But I fail to see how any construct can predict any sort of long term affect. I can only imagine the sheer mountain of data that would have to be collected and stored.

In the world of inventory analysis, companies like Target or Wal-Mart have insanely large server farms dedicated to inventory forecasting. They look at sales trends, manufacturing times, delivery delays, demographics, and over 100 other criteria for the sole purpose of attempting to keep the items you and I want to purchase on the shelves at the stores by our homes. These algorithms are constantly upgraded and evolved. The resultant computations run all day every day. The report goes out twice per week to vendors. Some vendors more times a week.

It is estimated that if the entire farm of servers were to go offline and need a complete system reset for 1 hour, there would be over 60 days of ill effect felt. That’s 60 days of back orders, shipping delays, manufacturing shortfalls, the works. Basically, it would cost either of these organizations upwards of 25% income loss for the affected period. That’s a massive amount of money. So, great pains are undertaken to ensure that these farms are not centrally located. The processing power is constantly upgraded and re-tasked. It is an enormous operation.

With all this and still, it is only 45% - 60% efficient given the time of year or month.

My point is that the amount of processing power and data storage behind predicting human behavior on a global scale is currently impossible even with 10 thousand Deep Blue's and 100 thousand Cray's running in tandem. It’s just impossible. I have been hearing about these "computer predictions" for years. It is a technological urban myth.

Computers are only as smart as the people who programmed them. That means if computers can predict the future, then India is about to rule the world.

 

[Rick Deckard]

Computers are only as smart as the people who programmed them. That means if computers can predict the future, then India is about to rule the world.

Actually, computers aren't smart at all.

 

[Seth]

It is estimated that if the entire farm of servers were to go offline and need a complete system reset for 1 hour, there would be over 60 days of ill effect felt. That’s 60 days of back orders, shipping delays, manufacturing shortfalls, the works. Basically, it would cost either or these organizations upwards of 25% income loss for the affected period. That’s a massive amount of money. So, great pains are undertaken to ensure that these farms are not centrally located. The processing power is constantly upgraded and re-tasked. It is an enormous operation.

I think David asked in a recent show and I am paraphrasing to great degree: “At what point does man and machine merge? Aren’t we at this point already with our reliance on technology and systems, if taken away, we are severely inhibited to the point of death?”…(sorry David, if that is nothing like what you said I’ll change it; that’s what I took from your statments)…Often I look around and see a fabricated society built on a precarious system of technology only a small few understand and control. Everything we interact with culturally is divvied out in subscription form or allotments few have the power to access without corporate or governmental assistance or approval. Isn’t this a fundamental downfall that has DRASTIC AND INHERENT COLLAPSE in big red letters written all over it? If this system fails are there enough knowledgeable individuals to rebuild the pieces?


*edit was to fix typo

 

[mike]

“At what point does man and machine merge?"

im of the view it was when we first learned to turn flint flakes into a knife blade, radical advances in this tech such as the spear and its even more sopisticated cousin the bow and arrows sealed our fate

all of these machines changed the very lifestyle and evolution of the species, from diet to enhanced migratory range, ie wearing of skins allowing you to live in colder climates.

im of the mind that once a bio form starts making tools, its on an unavoidable path to a point where the tools are making it.

man and machine merged when he stopped being just another feature of the garden and became the gardiner

 

[Ron Collins]

I think David asked in a recent show and I am paraphrasing to great degree: “At what point does man and machine merge? Aren’t we at this point already with our reliance on technology and systems, if taken away, we are severely inhibited to the point of death?”…(sorry David, if that is nothing like what you said I’ll change it; that’s what I took from your statments)…Often I look around and see a fabricated society built on a precarious system of technology only a small few understand and control.

This is not a new argument. The wheel, the bridle and saddle, the plow, the printing press, sailing ships, airplanes, the list goes on and on. Man is a tool builder. Man has been merged with his creations since the beginning of time. The plow is an easy concept, but could you build one? How about a saddle? How about a printing press? The point is that these are highly specific skilled professions that those at the time could scarcely do without. If every saddle maker in the world were killed 300 years ago, it would have been a major problem. But, just like computer systems and technicians/programmers today, the profession and product were too prolific. But, I would say that the total number of folks with expertise, real expertise, in this field is ever-expanding. I’m not talking about your neighbor’s kid that can build you a website or install the latest gizmo on your home machine. I’m talking about real techies. I think it is too prolific in world society, too engendered in University studies of wide and various majors to be able to vanish with no hope of fixing it if it were broken.

Everything we interact with culturally is divvied out in subscription form or allotments few have the power to access without corporate or governmental assistance or approval. Isn’t this a fundamental downfall that has DRASTIC AND INHERENT COLLAPSE in big red letters written all over it?

Not sure where you are going with this.

If this system fails are there enough knowledgeable individuals to rebuild the pieces

Arguably, this day is now. Without getting too technical, IP space has run out. Some have pushed IPv6 as a fix but this is only a band-aid. Bandwidth intensive information requests are on the rise and vast sections of the internet are already overloaded. But, you are still able to visit this site and interact with it.

This is because people are, every hour of everyday, altering portions of the internet for good or ill. Hardware, software, firmware are all being changed. Modern technology is a breathing thing. As much as saddles and bridles were once found in virtually every home in the civilized world so is technology.

In my opinion, it is this very prolific nature that renders computerized global analysis for any future paradigm impossible. The calculation would never end. There are no constants and the variables can never be locked down long enough to mean anything. It’s the Heisenberg uncertainty principle in practice.

 

[Seth]

Everything we interact with culturally is divvied out in subscription form or allotments few have the power to access without corporate or governmental assistance or approval. Isn’t this a fundamental downfall that has DRASTIC AND INHERENT COLLAPSE in big red letters written all over it?

Not sure where you are going with this.

I am saying EVERYTHING is compartmentalized including knowledge, energy, ideas, and skills. Independence is lost at the cost of technology and furtherance on our planet. And it seems intentional. You don't own anything solely. You require others to be successful. Everyone holds such a small piece of a huge pie in the way of knowledge and skill (some elect to let others hold their piece for them). Only as a group, can we proceed. This is not a good scenario for such a large population occupying such a small space. This scenario can possibly lead to a very large and dramatic collapse.

In this day and age it is possible to have the knowledge capable of building a rocket but not know how to grow one’s own food. It is possible to operate on the human brain in the morning and not have a clue as to how to change a tire in the afternoon. It is possible to lead one of the world's largest companies and still believe the earth was created 6000 years ago--- all compartmentalized thinking and action.

If this system fails are there enough knowledgeable individuals to rebuild the pieces

Arguably, this day is now.

Agreed.



*edit was to fix quotation error