Wednesday, April 2, 2014

Neural Networks and Change

This is a simplified map of your brain.

In my posting, Understanding Neural Networks, I mentioned how the brain is a neural network that is not "programmed" per se but rather, trained over time.  Now granted, parts of the brain are pre-programmed with certain routines - like how to breathe, digest food, flight or fight responses, and that sort of thing.

But much of your brain is programmed over time - trained - by your experiences.  When constructing a neural network to control a guided missile or whatever, you train the network by providing inputs, monitoring the outputs, and then providing correction to the network to alter the weighting values of the nodes, until you get the desired output.   Sounds a lot like Elementary School, doesn't it?

And that is because neural networks, in the Engineering sense, are modeled after the human brain.   Studying these artificial "brains" can tell us a lot about our own.   For example, it is perfectly possible to think you are teaching a neural network one thing, only to have it learn another.   In a famous example, experts thought they were teaching a missile guidance system to detect Soviet tanks.   Instead, they were teaching it to distinguish between spy photos of tanks (dark and grainy) and manufacturer's photos of tanks (clear and well-lit).   Teenagers work the same way.  You think you are teaching them one thing, but they learn another.

Over time, a neural network changes and adapts to its environment.  This is a survival skill.  As conditions change, the weighting factors in the nodes (and in the human brain there are billions of them) will subtly alter as the output of the network corrects the weighting factors to compensate for changed inputs, changed conditions, or changed desired outputs.   The longer the network is running, the more pronounced this change can be.

Another good analogy is the early primitive inertial navigation systems that were used by the military and are now used on airplanes and even some GPS units, to keep track of your location, between satellite updates.   An inertial navigation system uses accelerometers to measure acceleration in three-axes, and then integrates these values to find velocity in three axes, and integrates again to find distance.

In theory, if you know your starting point, and the inertial navigation system is 100% accurate, it should be able to track your location with extreme precision.   However, due to the double-integration needed to extract distance from acceleration (it's a Calculus thing) even the tiniest of errors is magnified dramatically.  Even with very, very precise accelerometers and measurement devices, an inertial navigation system can be off by yards within seconds - off by miles within minutes.

Thank God (or DARPA) that we have GPS instead, eh?

So what does this all mean?

Well, I got to thinking the other day how people change over time, particularly when you don't seem them for a few years or even a decade.   I related how my friend got into guns and accidentally shot himself recently.  They had these loaded guns in the house for "protection" when in fact they lived in a very safe suburb far from the crime of the big city.  And sadly, when I called him to wish him well, he doubled-down his bet, still claiming that having a loaded handgun around the house was a good idea, for "protection", even as he waited to be discharged from the hospital for a gunshot wound to the head.

The funny thing is, when I knew him "back in the day", he had no guns and would never have thought about buying one.   And far from being afraid of intruders, he would willingly drive to the worst and most dangerous parts of town to go to some night club or rave or whatever.   He had no fear then.  What changed?

Well, the truth is, we both changed.  Over the period of a decade-and-a-half, we moved on with our lives, found new jobs, spouses, and new places to live.  And our brains kept "training" on these new inputs.  Over time, they adapted to their new environments and assumed new values and new parameters.   He went from a hip young dude to a middle-aged man.  And the idea of a gun for "home protection" started to seem like a good one - to him - based on the feedback from his new peer group (who owned guns for "protection" as well).

It doesn't take long for the inertial navigation system to go far off course.   Before too long, you call up an old friend, and you are talking to a complete stranger.

And this is the norm, not the exception.  Because you see, I've changed too, over time.   Just since writing this blog (which is a form of feedback) I've changed my spending habits and philosophy.    A decade ago, I never thought how much things cost, just how I could go about getting them.  Today, I am much more cognizant of money and how hard it is to accumulate any.   And the rush of "owning things" burned itself out, over time.   For some folks, of course, this epiphany never occurs.

And these changes can occur fairly suddenly, as inputs to the network suddenly change - and desired outputs change as well.   As I noted in my Life Stages posting, there are certain breakpoints in your life, where things go from Plan A to Plan B.

The first and most notable for many people here in the USA is when they go from grade school to Junior High or Middle School.   In grade school, everyone gets along, and the idea of being "popular" is not as important.   In High School, young hormones kick in, and suddenly things change.   The inputs and desired outputs for the neural network shift, and some kids literally go insane.  It's called schizophrenia, and it is not hard to see why it strikes teenagers.  Someone changed all the rules, without telling them first.  The neural network doesn't adapt but instead breaks down, and "learns" a whole new set of really bad behaviors (which are rewarded with negative attention, causing a feedback loop in and of itself).

What got me thinking about this, in addition to my friend shooting himself (accidentally, he insists, although he has no recollection of the event) were recent visits with some old friends.   We've known each other for decades, and over that time, well, a lot has changed.   We are still friends, but we are now different people entirely.  This does create stress, when you visit.

And that is one reason I got off Facebook.   Keeping up with old friends is fine and all, but when one of them posts a diatribe against President Obama, you realize that they have changed dramatically over time, and maybe you didn't have as much in common as you thought you did.  It was more fun, back in the day, when you didn't know their political opinions.

And truth be told, friendships often form based on working at the same company, or living in the same town or some other contact-based indicia.   It seems you have a lot in common at the time.  But after you've gone your separate ways for a decade or more, well, things change, and you realize that you don't have anything in common anymore, and at the time, you really didn't have much to begin with.

That's OK.  Don't fret about it.  Trying to keep artificial relationships alive, decades later, is just stressful and bound to fail.

I wrote before about a friend of mine who goes to a "reunion" luncheon every year with girls she knew from the fourth grade.  Now, they are all pushing 50, and their lives have changed dramatically in the intervening four decades.  They go to this luncheon, and each tries to outdo the other with how successful their husbands are (or how successful they are) and my friend finds the whole event very painful.  But she doesn't want to be the first one (or second, anyway) to "drop out" of this event, so she keeps going, even though she hates it.   She does have a choice, of course.

My Dad decided to travel cross-country once, and visit his old friends from his college days, sleeping on their pull-out sofas.   Now, bear in mind that for him, college was sixty years ago.  I am not sure if he was trying to "reconnect" with old friends, or just being cheap and avoiding paying for a hotel room.   Needless to say, after the trip (which sounded like something from About Schmidt) he didn't mention these college buddies very much.   I suspect that the reunions were more awkward than anything, with the spouses wondering why a stranger is sleeping on their sofa in the den.

That is one reason I am not keen on School reunions.  After 20 or 30 years, you really don't have anything in common with the people you spent four years with.  They have grown up and moved on and their lives and their brains have changed dramatically.   Many folks go to these reunions to show off their success, or perhaps to relive the high school cheerleader fantasy.    The reality is, of course, that flaunting success in someone's face is a hollow experience, and the high school cheerleader is as fat as a cow.  Driving a Lamborghini to your High School reunion doesn't mark you as a winner, but rather a very pathetic loser.

So, then, what to do?   How do we combat this effect of the Neural Network and Inertial Navigation that forces us to drift apart over time?

Short answer: You can't.  Drift is going to occur because it is inevitable and programmed into our brains.  If we didn't change with time, our chances of survival would drop to zero, and we would not have evolved to the state we are in today.

Unless you are in constant contact with someone, on at least a weekly basis, chances are, the two of you will drift, over time.  Don't sweat it.  People waste more time and energy trying to "preserve" relationships that should be allowed to naturally and organically drift apart.  You meet new people in a new town or a new job, and they become the focus of your life.   Friends from before are doing the same thing.   You exchange Christmas cards and see each other once in a blue moon.   That's it.

Yes, some people do succeed in "re-connecting" with someone from years gone by.   But, I think this is more an effect of people establishing new relationships and then sustaining them by constant contact, than any significant re-connection.

For example, Carol Channing (yes, she is still alive) married her high school sweetheart after her abusive husband died.   They re-connected and formed a new relationship, not merely a continuation of their high-school romance.   Both of them had changed dramatically since High School, to be sure!

I think also that it is very hard to maintain an intimate relationship with more than a few people at a time.   Young people may have 400 "friends" on Facebook, but hardly know any of them.   And looking back, it seems that when I was younger, I had more friends, but the friendships were more superficial in nature.   Maybe as you get older, the relationships are fewer, but run deeper.   The downside to this, of course, is that your friends air all their dirty laundry with you - something that youngsters are loathe to do.

I am glad my friend is out of the hospital and while severely injured, at least looks like he will be able to care for himself for the rest of his life.   And I will go visit him and wish him well.   But I am realistic that we are not going to be the buddies we once were.   Too much has changed, over time, and the gun thing is emblematic of that.   He has a new life now, with new friends, and new, scary interests.

Sometimes you just have to let go.