Speech driven machines: Pretty easy these days

How do you make your machines respond to your verbal commands? Turn the lights on and off? Switch the appliances on and off? Query for the weather? Query for how many messages you have? No problem given modern machinery.

from pygsr import Pygsr
import os
import urllib2
speech = Pygsr()
while 1==1:
        speech.record(3) #Waiting for someone to call upon the machine
        try:
                phrase, complete_response = speech.speech_to_text('en_US') 
        except urllib2.HTTPError:
                print "No match"
                continue
        if "machine" in phrase:
                os.system('espeak  -s 155 -a 200 "I am at your command." ' ) #Prompt for the command
                speech.record(3)
                try:
                        phrase, complete_response = speech.speech_to_text('en_US')
                except urllib2.HTTPError:
                        os.system('espeak  -s 155 -a 200 "I did not understand." ' )
                if ("do" in phrase) and ("something" in phrase):
                        os.system('espeak  -s 155 -a 200 "I am doing something." ' ) #Do something
                elif ("end" in phrase) or ("exit" in phrase):
                        os.system('espeak  -s 155 -a 200 "Ending program." ' )
                        break
                else:
                        os.system('espeak  -s 155 -a 200 "Unknown command." ' )

How AIWorld6 works

How does the simulation work?

All agents have energy that they spend to do things and get by growing or eating other agents. When they reach zero energy they die. The abilities they have are: Moving, turning, attacking, sexual replication, and asexual replication. I’ve specifically chosen to give all agents the same abilities so they can only compete on how intelligently they use them.

How does the brain work?

All agents have a sparse, 2-layer neural network as their brain. There are 125 input nodes, 100 middle nodes, and 13 output nodes. The term ‘sparse’ means that not all input nodes are connected to all middle nodes and not all middle nodes are connected to all output nodes. Instead, each layer has anywhere between 1 and 500 connections.

All agents have the same inputs and the same set of possible decisions as outputs. The inputs are the values from the environment nearby the agent. For example, the difficulty of passing over the terrain directly in front of the agent, or whether or not there is another agent two spaces in front of and one space to the left of this one.

Let’s look at some example inputs. Here we see the values and what inputs they map to. For example, input node 59 would have the value 15, which is the value of energy of the agent at that location. And input node 34 would have the value 0.1, which is the cost of traversing the terrain at that location.

Let’s assume our agent has evolved a primitive brain that tells this agent to run away whenever it’s facing another agent. That brain might look something like the one below. Notice that the inputs which would show another agent being in front of this individual (57,59,62,63,64) all connect to an output that would be a decision to turn away. And that once the agent is turned away, the inputs which would show another agent being behind this individual (52,53,54) all map to the decision to move forward, away from that other agent. In this way, the brain is a basic decision tree for always turning and running.

Initially that agent was facing downward. When that agent chooses to turn right, it's perspective will shift.

The agent’s next decision, according to his brain structure would be to move forward. If the other agent was an aggressor, it would pursue and eventually corner the one running away. Of course if our running-away agent was had a better brain structure it’d either notice that it was going to be cornered (impassable terrain) or that it was bigger (more energy) and should counter attack instead.

That is just a brief look at some of the inputs and decisions agents could make. Of course, in the real simulation I don’t program any NNs myself, not even the initial ones; they evolve on their own.

How is time implemented in the simulation?

The simulation is iterative, rather than event based. Each iteration has two phases: decision and action. In the decision phase, each agent’s inputs are gathered, their NN is run, and the decision saved. This happens in parallel since no agent’s decision making process can affect any other agent, so there are no race conditions. The second phase is where each agent preforms an action. This happens in pseudo-random order and is single threaded since there are many race conditions such as moving into the same spot or attempting to eat each other.

Risks/Pitfalls I've encountered here: Previous versions of AI world used event based but the cost of maintaining the gradient of time was simply too computationally expensive for the value it provided.

One difficulty of running an iterative simulation is passing off the flow of control between master thread and worker threads without using a wait/sleep statement or any other choice that would have to be tuned. This was accomplished with used of 3 different mutex locks where master or workers were all either doing work or sleeping on one of the locks.

How does 'growing' work?

A given location will yield a certain value if an agent chooses to grow on it. The agent will only get this value, however, if there is no other agent directly adjacent when it chooses to grow.

Risks/Pitfalls I've encountered here: By only giving energy when no others are around, the simulation incentivizes against tightly packed agents. In previous simulations for whatever reason sustainable life didn't happen without them being able to get around each other. I do intent to test this rigorously now that I have a test framework.

How does 'attacking' work?

If agent A attacks agent B, it will steal energy in proportion to it's own energy from agent B, or if agent B has less than that, all the energy from agent B. For example, if the steal size was 10% and the efficiency of stealing was 50% and A had 100 and b had 100, after the attack A would have 105 and B would have 90.

Risks/Pitfalls I've encountered here: The simulation must kill agents as soon as they reach 0 energy and/or add a constant-value cost of attacking. In previous versions agents would attack each other and have an ever lower but still non-zero energy.

It also must have attacking equal to the size of the agent attacking, otherwise the total population of agents builds an ever increasing supply of energy and battles take too many turns to resolve. This is the only action agents take that's essentially different depending on anything but their brains.

How does 'asexual replication' work?

A new agent is created and given 1/3 of the energy of the agent that made it. The new agent's brain is a copy of it's parent except for any mutations (added brain connections or modified connection weights), which all happen according to a simulation-wide mutation rate.

How does 'sexual replication' work?

Similar to asexual reproduction except that the new brain is created by randomly choosing each connection from either parent's brain.

Risks/Pitfalls I've encountered here: In order to create a new, functional brain from two very similar and functional brains the connections must align. For example, if the 5th connection in brain A is from 35->56 with weight 1.2 and in brain B the 6th connection is from 35->56 with weight 1.21 then it's probably the same connection; you don't need two of them and you should have one in the new brain. For this reason, the replication code pays special attention to not creating accidental 1-off mutations that would bread the whole brain of the offspring.

It's also important to not make the sexual reproduction voluntary, which would be too much coordination for what little benefit it would bring. But one it's not voluntary, then it's only appropriate to take energy from the one choosing to replication. In previous versions of the simulation, the abuse of energy stealing in replication evolved rapidly and was the primary form of either replication or attack. In short, raping was common and I didn't feel it benefited the simulation.

Can they communicate?

At every run of their brains, some of the output nodes are special and their values are essentially saved to the location the agent is at. Other agents nearby can read the value at that location. I intend to prove eventually that they're using this for communication.

Risks/Pitfalls I've encountered here: In previous simulations agents were given a hash of the brain of agent's nearby. They could use that has to identify agent's like them. In practice this gave rise to look-alike agents, where some have a very different brain structure but it happens to hash to a successful agent. If agents were going to employ camouflage, I decided that the algorithm to hide or identify them should not be hard coded (as the hash was) and instead they'll have to give off signals and/or fake signals themselves. I suspect this will lead to more complex communications in the future.

Not giving agents any way to identify each other seemed like a mistake given the goal is more intelligent life and speciation has been common in previous simulations.

Causational studies of life

Poorly narrated video

A quick glance at AIWorld6

First, we see the pre-life world. Here we're seeding the world with agents that have random brain structures until we stumble upon a combination where an agent is smart enough to grow at least some of the time and replicate at least some of the time.

Here we see the first burst of life. This particular growth did not end up being successful in populating the world, but this is the first one to come close to sustainable life.

Many turns later we see the first evolution of war. Notice the agents in the center are spending a lot of time fighting while many around them are growing or replicating. Their brain-hash color is darker green, which distinguishes them as being related in brain structure and having different brains than the other colors. When all the other agents around this time are replicating some of them are ending up in light-green territory which seems to trigger for the first time some previously unused neural pathway to fight. After this moment, the percentage of time agents spend fighting goes from ~0.05% to 1-3% and the number of agents killed in fighting goes up dramatically, almost as high as those killed by starvation. This demonstrates the moment when these agents evolved the ability to defend their territory.

What is the goal of AIWorld6?

AIWorld6 is a tool used to perform causational studies of life and evolution. For example, a question we might ask is 'does the concept of aging cause a species to survive dynamic weather and environment patters better?'

Using only the tools of correlation, we'd have to look for situations where one part of a species did have aging and another part of that species did not. Or where two similar species demonstrated this difference. And then we'd have to look for times when they handled changing environmental conditions that were the same. And then we'd have to find many such examples that were so similar we could 'control' for every possible variable in order to prove a causational link. In short, correlational studies of this kind are impossible.

AIWorld6 solves that by creating a world where evolution is happening in real time and parameters of the world can be easily changed. In this world, all agents are created with equal bodies: same sensors, same abilities, etc. The only difference is their brains; this focuses all evolutionary changes to be on their intellect. By default all agents have a sparse 2-level NN with up to 1000 connections and no feedback loops (memory).

The combination of a simple world and a simple brain allows the simulation to run fast enough that evolution can happen in real time. Running for 4-hour on a 2-proc laptop yields not only the creation of initial lifeforms intelligent at least enough to replicate and grow, but also their first discovery of being able to attack and specific about *when* to attack so they're not blindly killing their offspring or related neighbors.

Now that the basic world is created and can run, the next steps are:

• Design tests that indicate intelligence. These will be the core measure of whether a species is more or less intelligent than another. (Remember that intelligence is the only factor they compete on so success as a species is equivalent to intelligence.)
• Design several tests, like introducing or removing communication, introducing or removing the ability to attack each other, etc.
• Create a long-running peer-to-peer network of the simulation. Today it runs as a single multi-threaded c program with a python UI. Running a long-duration, multi-machine network will allow us to evolve more intelligent creates for more complex tests in the future.

The new place

After 2 years of bullshit from various 'authorities' and departments, each taking their thousands of dollars in fees, we've begun digging at the new site. 1400 sqft of work space, here I come!

Brush knife

The GF and I have recently been doing some off-trail hiking. Though we've found we can get up or down slopes of 70deg and more, we get seriously fouled and stopped by thorny plans. Now, it's not my goal to ever be hacking things down in the forest, but it's just as stupid to get lost/hurt/dead because you've lead yourself into a head-high thicket of 0.5-1" thorns you just can't seem to escape. So I figured it was time to own a real knife.

In terms of requirements, I wanted it to be thick and durable enough to also double as an ice pick for climbing steep snow. I also wanted it to be stainless steel since I don't take good care of things. I wanted a hand guard, since I seem to always get thorns in my knuckles the other times I've used even a 2' machete. Finally, I want it to be small enough that I can draw it easily. (If you've ever owned a 2'+ sword and tried to draw one of those things from a back scabbard or tried to hike with one on a hip scabbard you'll see what I mean.)

Building the knife itself was pretty straightforward. I took a 1" x 3/8" flat 316 steel stock and cut it to have a 13" blade with enough room for a handle. I then took it to the bench grinder for maybe an hour to get a blade onto it. When I put my hand onto the handle and marked where my hand would be. Unlike a regular knife or sword, this one is only for me and thus it can fit my hand *perfectly*. I welded little chunks of stainless steel supporting my hand from below, above and on specifically for my thumb to hold onto. I then finished the blade with a metal file till it was sharp-ish. I'll eventually fire and quench it, but I'll have to find someone with a furnace first.

Building the scabbard was more challenging. In the end, I went with a series of 3D-printed sections all bolted onto a steel band I'd cut holes into. I'll eventually replace the bolts with rivets when my friend returns my rivet gun.

3D printing for me is still more of process than a push-button affair. At this point, makerbot stopped selling the 3mm wire mine uses so I'm buying shittier 3rd party stuff now. The machine itself also needs to be manually warmed in the control panel before printing. But it still does the job. In this case I made the sections 40mm tall with two mounting holes. Each section has knobs on the inside that hold the blade on either side and don't let the edge touch the scabbard.

Spring gun prototype is done!

The spring gun is fully built. But it functions at ~12% efficiency, which is lower than I hoped for. I suspect this is because of losses in friction between the spring and the buffer tube that it's around. I tried oiling it but this produced no better results.

The pictures here are it shooting through my cronometer. As part of this, I learned that the crono requires it's backdrop/shield to be back-lit and expects this to be from the sun. Indoor it doesn't work with neon lights because the crono can see the 60Hz of those things. It turns out finding incandescent lights needed for the crono was harder than expected since they've become banned. So I just bought then from the manufacturer for a million dollars. Not sure where they get them.

In terms of performance we're expecting, we have to find the energy the spring will release: I measured the displacement of the spring with a 10lb (44N) weight and found it had about 2" (0.0508 m), which results in a K (spring constant) of 44/0.0508 = 886. The current setting is for it to release when it gets to about 3.75" (0.09525 m), which should yield 0.5 * 886 * 0.09525^2 = 3.929J

When fired, it launched a 0.5" ball bearing at ~35ft/s (10.67m/s). The ball weighed in at 8.9g (0.0084kg). It then must have had 0.5 * 0.0084 * 10.67^2 = 0.4782J. This means the machine is operating at 12% efficiency.

The next step is to check the efficiency at different release levels. If efficiency goes down as power increases that would be a bad sign. After that, it's time to try other springs.

Spring Gun progress so far

I'm making progress, probably about 2/3 done with the basic prototype. The goal of this project is to make a prototype that I can experiment with, not something I expect to carry around. That means weight is a second priority to reconfigurability.

Because the device I'm making is small and complex, precision of parts is more important than any other metal-working project I've done before. I considered actually 3d printing many parts but concluded that in many cases they would not be strong enough to handle the forces involved.

In practice, my digital calliper and the milling-machine's measures were invaluable. I calculated the size and position of each part/cut/hole at the level of 1/16" and in practice I was able to measure that each was placed within 1/32" with the exception of welding. It's amazing how significantly a piece of metal will warp during welding. I have not yet figured out how to solve this.

As noted, this involved a lot of milling. Among other things, I found that ~300-400 RPM on the mill was a good speed for steel. At lower speeds I think my mill takes too much torque and sometimes vibrates significantly. At higher speeds it needs more cutting oil and seems to produce more metal slivers instead of metal ribbons.

The act of welding it required in many cases placing parts on top of other stand off pieces. In this sense I found it very helpful to have a variety of shims: 1/8" flats, 1/4", 1/2", 1" square bars.