deep neural network machine learning

[Neko]

anyone following http://www.wired.com/2016/05/google-alpha-go-ai/ the fact that computers now can beat the best humans in a game where they cannot compute all possible outcomes?

This deep neural network learning sounds really cool...

[Neko]

you can train it to play mario kart

YouTube

[Darkflame]

I posted this on the tech thread, but it deserves its own.
Google open-sourced a lot of it.
One significant part of it is TensorFlow, which is how this stuff gets trained;
https://www.tensorflow.org/

(Currently used for both their speech and image recognition as well as a few other things)

When I say image recognition, incidentally, I mean "it can recognize objects in a image"

https://youtu.be/eve8DkkVdhI

https://www.raspberrypi.org/blog/ras...ion-google-io/

Oh, and you can technically use this right now yourself;
https://cloud.google.com/vision/

However, its not free. They will ask for your credit card information even for a demo. (which sort of makes sense - as I can imagine this is probably quite intensive work on the server side so they cant just give it away or make it advert based easily).

[Neko]

do you think we can use tensorflow to learn how to play games?

that would be teriffic

[Darkflame]

Maybe.
My understanding though is once you train a AI in this way its actually very hard to work out exactly how its working.
Remember those "deepdream" images that were popular a few months back?

http://deepdreamgenerator.com/

That was a incidental result of Google trying to figure out how its image recognition AIs were actually recognizing stuff.
Essentially they tried to run their AI backwards.

[Neko]

yes! that is kinda scary. It's just pure math you're looking at. You cannot see the 'code' as there is no 'code'.


I really want to learn this, but I don't want to just randomly start studying math without knowing what basics and subjects you need to study.

I know programming, but this is a really different world

[Darkflame]

It is fairly different. Nothings definitive anymore,the logic isn't absolute anymore.

However, from what I have seen glancing at the Tensorflow tutorials, its actually very flow-chart based (as in, literally a flow chart editor).
I think a "programmers mind" might be enough to get started, although thats just a guess as I havnt had time myself to look deeply.

[Lupin]

I've been looking into deep learning for a while now, and I think I have a good grasp of the general picture.
The best resource I found so far is this online book: http://neuralnetworksanddeeplearning.com/
There's definitely a bit of math there, but it's quite manageable.

I also find the TensorFlow playground useful to help visualize what's actually happening.

TensorFlow is actually not bringing much new stuff to the table, it's main value is in providing a framework that works both for production and research, while most other older frameworks are research-oriented. It's also got better tooling. But the important stuff is in the math, not in the particular framework you use.

There are 2 important things to grasp in my experience to get a good grasp of the whole picture.
The first is the idea of Machine Learning in general and particularly the role of the "optimization/cost function" in learning.
The second is to get a good intuition for neural networks (both biological and artificial ones). NNs are not new stuff, even Deep Neural Networks, and they are a quite straight-forward model of what's inside our actual brains.

The field is quite complex, definitely not something you can learn overnight. But it's moving very very fast, and I'm pretty sure we're facing a revolution of the same magnitude as personal computing or the internet.

Particularly, the last developments from DeepMind (Atari games, AlphaGo) are a very significant step towards artificial general intelligence (AGI).
Self-driving cars, the Falcon 9 landings, as well as many huge tech successes of the recent years all gravitate around similar ideas and techniques.
The DeepMind paper about Atari games is mind-blowing, even if you don't go through the math in details.

Heavy progress seem to be made when there's a cross-pollination of ideas between state of the art neuroscience and comp-sci. We're in for an interesting decade.

[Polaris]

Wow that's impressive stuff...
I don't have much to say other than please keep talking about this '

[Neko]

Originally Posted by Lupin

I've been looking into deep learning for a while now, and I think I have a good grasp of the general picture.
The best resource I found so far is this online book: http://neuralnetworksanddeeplearning.com/
There's definitely a bit of math there, but it's quite manageable.

I also find the TensorFlow playground useful to help visualize what's actually happening.

TensorFlow is actually not bringing much new stuff to the table, it's main value is in providing a framework that works both for production and research, while most other older frameworks are research-oriented. It's also got better tooling. But the important stuff is in the math, not in the particular framework you use.

There are 2 important things to grasp in my experience to get a good grasp of the whole picture.
The first is the idea of Machine Learning in general and particularly the role of the "optimization/cost function" in learning.
The second is to get a good intuition for neural networks (both biological and artificial ones). NNs are not new stuff, even Deep Neural Networks, and they are a quite straight-forward model of what's inside our actual brains.

The field is quite complex, definitely not something you can learn overnight. But it's moving very very fast, and I'm pretty sure we're facing a revolution of the same magnitude as personal computing or the internet.

Particularly, the last developments from DeepMind (Atari games, AlphaGo) are a very significant step towards artificial general intelligence (AGI).
Self-driving cars, the Falcon 9 landings, as well as many huge tech successes of the recent years all gravitate around similar ideas and techniques.
The DeepMind paper about Atari games is mind-blowing, even if you don't go through the math in details.

Heavy progress seem to be made when there's a cross-pollination of ideas between state of the art neuroscience and comp-sci. We're in for an interesting decade.

I think i need to update my math

[Darkflame]

The field is quite complex, definitely not something you can learn overnight. But it's moving very very fast, and I'm pretty sure we're facing a revolution of the same magnitude as personal computing or the internet.

Agreed. In fact, I think any child or student right now would be wise to consider learning this stuff with a mind to future careers. Its going to become the main method to get non-trival computer tasks done I think.

Of, course, it goes the other way too; Id advise people to get into bio engineering. Coding gene strands to create custom organisms. There is strong analogies between bio-engineering and early computing right now.

Heavy progress seem to be made when there's a cross-pollination of ideas between state of the art neuroscience and comp-sci.

Very much so.
But needs to be a little bit of caution as to what we already know. I see some news/blogs etc inferring the main difference between the brain and computer neural networks is merely numbers.
But even a single neuron we dont fully know how to simulate so its not just scaling.
I think it was only in the last few years we have started to look at the mechanical effects as well as the bio-electrical ones.

[Polaris]

Your first post reminded me of a video I found a while ago, Jesse. The AI also learns playing (this time on a NES), but the guy has an algorithm which lets his AI play different games without the need of modifying it.
https://www.youtube.com/watch?v=xOCurBYI_gY

[Homeless]

Originally Posted by Polaris

Your first post reminded me of a video I found a while ago, Jesse. The AI also learns playing (this time on a NES), but the guy has an algorithm which lets his AI play different games without the need of modifying it.
https://www.youtube.com/watch?v=xOCurBYI_gY

That was lovely, thank you for sharing.

[Polaris]

There's three episodes of that, the AI plays a shmup absolutely amazingly in the third one !

[Neko]

Originally Posted by Polaris

There's three episodes of that, the AI plays a shmup absolutely amazingly in the third one !

will check it out tonight!

I am planning to train the network shown in video 1 to play ghostvalley course

edit: looks like the mario kart version of Mari/o isn't open sourced :\

[Neko]

Got the mari/o program to work

[Darkflame]

cool

[Neko]

So I have a generation that knows how to play level 1. It is currently training level 2. I wonder if it still can finish level 1 when it can finish level 2.

[Polaris]

You mean that's downloadable ?! I need to check that out !

[Neko]

yeah you can. but the inputs are not equal to what we see. it sees all enemies the same regardless of state, type and size. that doesn't seem fair.

So my next idea is to alter the script so it gets the right input.

[Neko]

I wonder if we could alter it also so it runs in parallel. Now it goes too slow. You would want to run an entire generation run in parallel

[Polaris]

May I ask where you downloaded it ? There's a link to the source code under the video, but I don't know what to do with that

[Neko]

1. download bizzhwawk emulator
2. save the sourcecode in a file called neatevolve.lua in the lua/snes folder
3. get the amarican super mario world rom
4. save a state
5. save the state file in the lua/snes folder and rename it DP1.state

bingo!

[Polaris]

You're the best

[Styx]

Originally Posted by Jesse

1. download bizzhwawk emulator
2. save the sourcecode in a file called neatevolve.lua in the lua/snes folder
3. get the amarican super mario world rom
4. save a state
5. save the state file in the lua/snes folder and rename it DP1.state

bingo!

Originally Posted by Polaris

You're the best

Did I miss some context or what?