Are larger neural networks stable?

February 2nd, 2016

So, as we approach the singularity – and all indications are that in about 15 years we will be able to build a mind bigger than ours, if Moore’s law holds – one interesting question is whether a larger neural network than us would be stable.

This is a subject that, if Google is to be believed, is of much scholarly interest. I’m still not at a place to evaluate the validity of the discussions – I’m still working my way through a full understanding of neural coding – but I think it’s a interesting question to be asking.

One presumes that some sort of optimization process took place (either via evolution or design – or quite possibly both) in determining how large the human mind is – but whether it was a decision about stability or a decision about power consumption remains to be seen.

In a neural network of fixed size, it seems clear that you have to make some tradeoffs. You can get more intelligence out of your 10^11 neurons, but you will likely have to sacrifice some stability. You can also make tradeoffs between intelligence and speed, for example. But in the end, humans in general all have the same number of neurons, so in order to get more of one aspect of performance, you’re going to have to lose some other aspect.

When we start building minds bigger than ours, the question that occurs is, will they be more stable? Less? Will more neurons mean you can simultaneously have a IQ of 2000 (sorry, Holly!) and be rock solid, stable, and reliable? Or will it turn out that the further you delve into intelligence, the more the system tends to oscillate or otherwise show bad signs of feedback coupling?

Only time will tell. As the eternal paranoid optimist, my hope is that we will find that we can create a mind that can explain how to build a much better world – in words even a Trump supporter can understand. But my fear is that we’ll discover we can’t even build a trillion-neuron neural network that’s stable at all.

We also have to figure out how we’re going to treat our hypothetical trillion-neuron creation. Clearly it deserves the same rights as we have, but how do we compensate it for the miracles it can bring forth? What do we have to offer that it will want? And if we engineer a need into it so that it will want in order to have that need met, what moral position does that leave us in?

Neural networks in output mode

January 31st, 2016

So, one of the common threads of the last few years has been me considering the possibility that nothing I am experiencing is happening to anyone but me – or possibly, just a subset of what I am experiencing is happening to only me, while other bits are happening to everyone. Certainly, questioning how much my conscious experience has to do with the data coming at me.

One of the bits of research that really underlined the validity of this was this. In essence, researchers discovered that artificial neural networks configured for image recognition could produce *output* that was related to the input they were trained to recognize. If you needed a larger neon sign that what you’re experiencing might not have that much to do with what’s coming in on your senses, I don’t know what to do for you.

As my experience polarizes further and further towards smart and dumb and love and fear I get more and more hints about the underlying patterns. And more and more food for thought about what experiences might be coming from where.

One thing I’ve definitely experienced is memory alignment issues. One of the reasons I keep this journal is so I can go back and read it and check to make sure what I remember and what I talk about is the same. A force working against that is that it’s hard to honestly talk about things that went wrong in my life, and so back in the day I didn’t. This is something I’ve changed a fair amount, but it is scary – especially when I see things like facebook banning sheer.us, although I’ve decided after careful consideration about what facebook is that that is a compliment.

Yes, I’ve apparently finally achieved being a true radical, rather than the political equivalent of a script kiddie. I’m starting to have alternate suggestions about how to do fundamental things. It may be that none of them are any good, but it may also be that the only way to find out is to simulate them. One of the exciting things about seeing the singularity (a mind bigger than a human’s) rushing up at us is that if we can make friends with a trillion-neuron mind (which may be a challenge) we might be able to get some real answers about what the best configuration for the world might be. That’s assuming a trillion-neuron mind is even stable, a subject I hope to write a article about soon.

Testing LJ crosspost

January 31st, 2016

Note that crossposting has been disabled for a couple of years.. go to my blog on sheer.us if you want to read the events of those years.

Obsolescence

January 31st, 2016

So, with the singularity apparently about 15 years away, I find myself pondering the question of why am I here and what am I good at in a different light.

The only meaningful answer I can come up with is to experience things from my point of view. I have no doubt a artificial neural network that’s bigger than I am can write better music, better text, better code. But it can’t *experience* in the same way I can – I don’t doubt that it can experience a conscious experience, but it’s going to be *different*. I think. It’ll be hard to even really find out the answer to that question, but for the moment I assume what I bring to the table isn’t so much intelligence as it is a particular, unique flavor.

One thing I’d really be curious to find is someone else with a blog similar to mine. I feel a lot of the time like I’m pretty unique, but perhaps there are in fact millions of people like me out there. (Although you would think if there were, capitalism would have died a honorable death, replaced by something that worked better, by now)

I actually sometimes think capitalism would work beautifully, if everyone understood the money had no value. That it’s not the basic system that’s flawed, but rather the set of ideas we’ve built up on top of it.

But I remind myself of the great depression. And what’s impressive to me about the great depression is there was no shortage of steel, or copper, or food, or power. The shortage was of money flowing. And we accepted that.

Sometimes I think humans are entirely too caught up in the rule of law. The sexting teens being arrested are a impressive example of this, but there are tons of examples. We think A: we need to make rules and B: we need to punish people who don’t follow them, even when they were stupid rules.

But then, I’m not the average person. I read the bible saying to stone gay people and know, this isn’t the work of a higher power and never was. Others read it saying that and say, that’s god’s word, we’d rather our children commit suicide than change our minds about that. (I’m looking at you, Mormons.. )

Anyway, back to the original topic. So, I don’t think I will be obsolete even when there are life forms more advanced than I am, because I don’t think they’ll be able to experience the world the same way I do. Now, granted, I’d really rather be experiencing a much better world, which is part of why I like the idea of there being life forms more advanced than I am – it’s possible that if we build something with a trillion neurons, and it explains to us how dumb our economic system is, we might just listen. Or perhaps it’ll explain to us that it’s absolutely perfect, and then it’ll explain why in a way that can reach me, and I’ll no longer feel like my friends are constantly barely making ends meet mostly because we built a badly designed world.

Rights for electronic life

January 30th, 2016

So, recently I ran across this.

My first reaction was, holy shmoo, the singularity is almost here!

Actually, there’s all kinds of interesting problems here. I’ve talked with a number of my friends about the question of whether, if we created a accurate software model of a human, it would exhibit free will. It’s a really interesting question – if the answer is yes, that’s a serious blow to theology but a major boost to the rest of us.

But there’s a natural side question which comes up – which is, supposing we can get the neuron count up from a million to a billion per chip. If moore’s law were to hold, this would take – let’s see, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024 = 11 18-month cycles. At that point, making a 100-billion neuron mind out of the chips becomes practical. Said creature has as many neurons as we do – but is it a person?

My guess is, legally, initially, no. In fact, we’ll probably see all sorts of awful behavior as we debug, including repeatedly murdering the poor thing (turning off the power, over and over).

We may even see them turned into slaves, although I really hope we’re beyond that by now. I don’t mind enslaving small neural nets that will never show free will or understand suffering, or enslaving turing machines which are incapable of a original thought, but the idea of enslaving something that’s as capable as we are is disturbing.

At some point, however, we’ll have to acknowledge that a person’s a person, no matter what they’re made of. I see signs we’re moving in this direction with India granting personhood to dolphins (about bloody time!) and I have hopes to someday see it granted to any individual who can pass the mirror test. (If you know you’re a person, then you are)

It does remind me of “Jerry was a man”. It’s a question we’ll have to wrestle with – I hope we haven’t gotten so locked into the idea that electrons just do what we tell them to with turing machines (where that’s true) that we can’t realize that if we build a sufficiently large neural network out of transistors, it has the same rights that we do – in fact, ‘birthing’ might be a better phrase than ‘building’ here, since we are undoubtedly creating a new life form.

There’s all sorts of interesting corollaries to this as well. If we succeed in building something self-aware out of transistors, our race will be experiencing first contact. Granted, we’ll have *built* ET instead of met him out there in the sky, but that doesn’t change the fact that it is first contact. A life form made out of silicon is likely to be *different* – have different values, enjoy different things. This has been explored quite a bit in science fiction, but it was completely news to me that I was going to see it in my lifetime (assuming the actuarial tables describe me) as science fact.

If we build something 100 billion neurons in size and it’s *not* self-aware, this also has interesting implications – it asks the question “Where is the magic coming from?”. This outcome would also be incredibly cool, and lead us off in another, equally interesting set of adventures.

There’s also the question of the singularity – what happens when we build something with 200 billion neurons? There’s another article I keep meaning to write about intelligence and stability, but one interesting thing I would note is that plus or minus a few percent, all humans have the same 100 billion neurons, therefore increased intelligence or performance in our minds comes from changing the way we connect them. It’s possible that a larger neural net won’t be more intelligent at all – or that it will be completely unstable – or that it will be much, much, *much* more intelligent. All of us are going to be curious about what it has to say, in the latter case, and in any case we’re going to learn a lot of interesting things.

However, I do think we should all sit down and talk about the ethical issues *before* we build something that should have legal rights. I think we probably will – this has been addressed in numerous forums so it’s undoubtedly something people are aware of. One of my favorite Star Trek themes, addressed numerous times in TNG.

Me and Facebook – end of a era?

January 30th, 2016

So, Facebook thoughtfully provided me this response:

“Hi,

Thanks for taking the time to submit this report. While we don’t currently provide individual support for this issue, this information will help us identify bugs on our site.

In other words, you’re censored, no, we won’t tell you why.

Well, Facebook, goodbye. I have enough emotional literacy to know that I don’t really enjoy visiting you that much anyway, and now that you’ve started censoring me, I’m not interested in continuing. The nice thing about running my own web server is that I’m fairly sure I’m not going to censor myself.

To the rest of you all still on the site, consider leaving. There’s a lot of better content out there. Facebook has turned into a collection of games you will play while feeling totally numb and everyone repeating memes instead of talking about their lives – I guess nothing lasts forever.

Ask yourself this question – how do you *feel* while browsing the site? Do you not think there are things you could be doing that would feel better? Like, you know, actual social interaction with real humans, face to face? Or, in my case, spending more time writing and recording music? Of course, no one will hear it, since I’ve got nowhere to post it any more. 😉 Well, I did talk about creating a mailing list, I guess I will go do that.

Banned by facebook

January 29th, 2016

I’ve been banned. I ran a malware scan (remote) of sheer.us, and checked with google, and no, I am not exporting malware to the world. It’s not that I’ve been p0wned.

I’m curious which of the many possibly-undesirable-to-facebook things I have talked about it is.

I can’t decide whether to be angry or honored.

Why I started having doubts about a simple file permisison based universe, and what could be done to improve it.

January 29th, 2016

So, originally I was in favor of a simple file-permission based model for protecting us from each other. I.e. there would be various flags (see/touch/telepath/whatnot) and each of us could set and clear them for individuals and groups.

The sentence that made me rethink this was me wanting to restrict all members of the US government from communicating with me. It was undoubtedly a emotional overreaction to yet another police shooting, combined with making the mistake of googling ‘is the US a police state’.

Now, I think this might be one of those rare examples where adding some complexity would help. The first thing I’d add is a timer with a maximum value – i.e. I can block the US government for 5 minutes, or a year, or five years, but not indefinitely. A second radio button would specify whether at the end of the timer I should be asked if I want to clear the bit, or if it should just clear. More invasive bits should be on a default-deny rather than default-allow – i.e. I’d like to have to set telepathy on for you before you can read my mind.

I do however always worry – any time a design requires more complexity, I suspect it’s not the right design. I don’t know what the right answer is, though. But while I think a file permission universe would be a interesting place to visit, I am not yet convinced I’d want to be locked in it for eternity.

One of the questions I’m asking is what sort of universe could you enjoy forever? What *possible* option wouldn’t end up as torture?

Religion

January 29th, 2016

It’s so odd to me that I managed to get hurt so much by people who had such good intentions. I guess I talked about that in a previous post. I tried to explain to a friend of mine the other day what parts of Christianity not to teach his kids. I wonder if he listened, and if so, if he thought about what I said.

For those of you who are curious.. don’t teach your children that they are so flawed that someone had to die for their failings. Because A: you don’t know that and B: it’s a very damaging message.

There’s so much about Christianity that makes me roll my eyes at this point. One of the most impressive bits of stupidity is the idea that God can’t arrange to incarnate any time *e wants. Only if we have a impressively incompetent God. The bible claims God’s voice would deafen us – while at the same time claiming God is omnipotent. Look, omnipotence includes a gain control, last I looked. The glaring silence, and competing religions all claiming to be right, makes the most likely hypothesis seem like there isn’t anything you’d call God – like we’re doing this bottom-up, not top-down.

For that matter, it’s hard to even fathom how a creator who can’t even figure out advanced error correction could be called perfect. (Cancer happens because DNA uses a simple checksum instead of something more advanced like CRC-32) You’re in a awkward place if you’re defending that your diety is a intelligent designer, but they don’t have I.T. as good as we have on Earth. And very little in the bible makes me think it was written by anyone who had seen as far as we have. If you look really hard on the horizon over there, where computers and humans dance together, you can see something I might call heaven. We can get to a really awesome world from here. A place where no one is hungry, where everyone can have any experience they want (although experiences hurting other people in ways no one wants to volunteer to be hurt are going to have to be simulated rather than live)

If you’d wanted to impress me with God’s all seeing ways, the book would not have contained the line “don’t improve me”. That just impressed me with how much it was not the work of a higher power.

Actually, the old testament did a great job of that, too. God has such a fragile ego that he has to talk about himself as the LORD in all caps? Oh, yes, he wants your money, too. That the priests are the ones writing the book has nothing to do with that at all.

And he was such a loathsome creature. Stone someone because they like to have sex with the same gender? How can we read that and not just go, this is bullshit, and throw the book out? Why would ANYONE expose their children to this mess?

It does at times make me wonder, could I write a better religion? But I refuse to be Joe Smith, or even L. Ron Hubbard. I do sometimes think I need to write a better one just to load into myself.

Why do I write anti-christian articles? well, at this point, because I think it’s healthy for me to do so. It’s helping me unload the religion to point out how badly written it is, and how it’s not something anyone should expose their children to. Among other things, children have huge numbers of unallocated neurons. Do you really want them using some of them to imagine a creature who specializes in evil? (It’s a sign of my impressive disdain for the religion that I’m not sure when I say this whether I’m talking about God or Satan)

I would rather, if I’m going to believe in a God, believe in a God that is better than I can possibly imagine.. and I can already imagine a God that is better than anything in that book and still fit it with the experiences I’ve had so far on Earth. Assuming certain postulates are true, such as this place is deliberately bad so we can come here when lotus-eating in amazing-land pales, or when we want to take on a challenge.

Christians in general believe in a incompetent God. For example, why would a competent deity ever connect a soul to a body *e knew wasn’t going to ever walk on the earth? When I told my mom this she asserted that “It doesn’t work that way” – to which my response is, and you know this *how*? Because a book that advocates keeping slaves and murdering people told you? Let’s have a little talk about virtualization and some educated guesses we can make about what a intelligent, capable deity would do..

Now, I admit it, I’m angry. I would love for all the abrahamic religions to be studied as curiosities, rather than read as anything anyone would want to believe in. I would love for them to meet my old friend the low level format tool, for that matter. The Unitarians, I can get behind. Short source code. Hard to screw up. But I’m not a universalist.. I don’t buy that religion is a universal thing, or something we need, or even want, for a lot of definitions of the word religion. The idea that a belief can get wedged into our neural net so thoroughly that we hold onto it even if it fails reality testing, even if it fails common sense, I find far more scary than desirable.

And the idea that God would torture me for all eternity because I couldn’t pick from a plethora of religions that all looked highly questionable? I find that one just plain absurd. It’s certainly not even remotely compatible with the idea of God being a loving entity as I understand love.

Then again, who would I torture eternally? No one. Not even Hitler.

I do have second thoughts about my file permission based universe from time to time. It’s definitely not something I’d sign up to be locked in forever. My fear is that ultimately we would all end up completely isolated from each other. But it’s a appealing idea, nonetheless. The thing is, I haven’t met anyone that I’d actually lock out of my world yet, although I’ve met people I would like to maintain a fair amount of distance from. Two of them, so far.

But I’m not sure what i *would* sign up to be locked in forever. It would have to be one hell of a system. More degrees of freedom than I could imagine or understand. Really good online help.

It’s kind of moot, since at the moment my life rates as ‘acceptable’ occasionally bordering on ‘good’ and I’m still working like hell to try to get it to ‘great’. Despite all my anger and frustration at the world, the truth is most of the problems are internal. I think a lot of them were *caused* by my experiences in the world, but I don’t think the world is hurting me any more, for the most part. And I’m learning to react to people in the world hurting me by changing my trust level of those people, which leaves them less openings to hurt me in the future.

January 29th, 2016

It’s a challenge sometimes. When I think about the fact that almost certainly the last three presidents all used drugs, but none of them issued a block pardon to all drug offenders arrested for no crimes other than possession / nonviolent distribution.

I do feel a bit like we live in, increasingly, a police state. There are way too many laws, they’re way too complex and poorly written, it seems lately it’s acceptable for a cop to shoot you just because he doesn’t like the way you looked at him and he will not lose his job nor his right to carry and continue shooting innocents.

Trump feels to me like a page right out of Hitler. The same sort of hate. Why can’t we make our government stop, have a constitutional convention, make a prioritized list of everything that sucks, and figure out closed-loop results-oriented ways to make it not suck?

Part of why is politics involves some sort of insanity in humans. I swear neither party wants to hear from me. They want my money, but I am the last guy they want to talk to. I’ve sent Bernie many emails asking to just have a staffer talk to me about my ideas concerning resource allocation – even a 21 year old intern – just because I think I could convince one guy enough that I’d have a dog in the fight – and no response. In the case of Hillary and Trump I didn’t even bother because, yes, what’s the point? They both have huge things to gain from a better resource allocation system but I am guessing neither one of them knows that or is willing to find that out. I’d love to be proved wrong. I really want things to improve *massively* for people here. I just .. don’t see it. I want to see it. I’m ready to see it. But so far..

And I will admit, I have a hard time seeing the point of view of my political opponents. I just can’t wrap my head around it at all. I try, but it basically feels like they just can’t make the intellectual leaps I have and so they want me to be forced to live in a country that’s stuck at the level they can understand and believe in. And I have several people I love who are republicans.. I love them, but I don’t understand them. I’m sure it’s not that these are stupid people. I’m sure there’s another reasoning chain I could have gone down that makes their views make complete and total sense. But I didn’t go down it, and un-learning what I know now would be a challenge.