DEEP LEARNING REVOLUTION, summer 2015, state of the art & topnotch links

deepLearningAI500➡ ⭐ 💡 ❗ 😮 😎 😀 ❤ hi all. did you ever get the feeling like something big might be flying by fast and youre missing it? a sort of tech “FOMO” (“fear of missing out”?) got that feeling years ago with the web. am feeling it a bit lately wrt DEEP LEARNING.

my last deep learning post was 3rd quarter 2013. am not going to downplay it/ mince words, it was prescient and way ahead of the curve. almost perfectly timed it, only a few months later (late Jan 2014) google bought deepmind![a1][d10][d11][g4] apparently largely/ alone on their groundbreaking work on video game learning. this was published in Nature early this year building on the Dec 2013 paper.[e9] (again shortly after my post and more evidence of my impeccable/ nearly perfect timing on this one!) 😎 😀

this is something like the gunshot that started the ensuing race/ stampede/ melee, now in full sway. facebook jumped into it[a3][a4][a5] and there was much news of other startups.[d] experts in the field, which are rather rare, are in high demand.[d1] there has been massive media coverage [b] by top outlets in the MSM eg Wired, MIT tech review, NYT, american scientist, new yorker, guardian, dailymail. google seems to be significantly ahead of everyone but how long can that edge last? even MS jumped into the fray.[d8]

there are a few top experts frequently cited. Hinton, LeCun, Bengio, (aka “the canadian mafia”), & google’s ringer Ng.

however there is other massive white-hot news/ advances/ breakthroughs on image/ object/ face recognition that is fueling constant headlines.[k][l][m] long ago in my college days, this was a science fiction thought that AI could ever achieve object recognition within my lifetime. science fact, now! breathtaking! revolutionary![h] and there are strong combinations/ crosspollinations of this field with the other big wave long noted here, Big Data which are part of its massive momentum.

one of my big questions is, how is this going to be monetized? the big giants Google & Yahoo seem to be wasting no time. facebook already has a lot of face recognition built in, and presumably Google is going to plug in object and face recognition into its search engine ASAP (but so far despite all the frenzy maybe neither have this technology in production systems yet?).

the image results led to two major spinoffs. one is so called “adversarial techniques” which find weird noisy images that cause the classifications to fail.[f] the other is “inceptionism” or “dreaming”[j] where the networks are used to generate or modify images. these seem likely to be the introductory volleys in entire new areas of longrunning research.

there is much here to delve into, there are basic tutorial-like introductions in blogs and presentations,[c] amazing papers,[e] many profiles & interviews,[g] open source code/ pkgs.[i]

(it has been quite a challenge to track all this and my cup runneth over, the net is full of fish, or perhaps the web is full of flies. planning another followup post on lots of AI advancements that are quite substantial also, but not fitting into the deep learning category per se.)

viva la revolution!

💡 💡 💡 PS one idea that has been occurring to me last few years, and last few months. have long been interested in the real nature of AI and studied it deeply. this could take an entire book, but my “novel” theory is that its fundamentally oriented around novelty detection. existing algorithms hint at this but nobody has fully seized on this idea yet afaik/ct. novelty is difficult to measure/ quantify but recently another idea occurred to me: how about measuring novelty as the amount of signal that the neural network can extract? this would work for non-fixed sized networks that “grow” (eg add neurons) as they find new signal. the system needs to know when current neuron weights seem to have stabilized and then add new neurons, and if the new weights of the new neurons do not fluctuate randomly, then voila, new signal has been extracted! this seems to me like a very, very substantial idea & intend to write more on it in the future.

PPS changing to a new font to celebrate wordpress making the feature available for free, and today a record number of daily hits lighting up this post! thx very much reddit, google+, hackernews, vk, facebook, stackexchange! so great to connect with a lot of deep learning geeks/ hackers out there! am thinking will now celebrate new milestones/ shifts/ moods on the blog by switching fonts!

a. deep learning
b. intro
c. edu
d. corp direction/ strategy
e. papers
f. adversarial
g. profiles/ interviews
h. revolution
i. code/ pkgs/ open source
j. dreaming/ inceptionism
k. object/ image recognition
l. facerec
m. captioning/ labelling

1 thought on “DEEP LEARNING REVOLUTION, summer 2015, state of the art & topnotch links

  1. Simon Hughes

    I’ve had a very similar thought regarding novelty. I believe the brain learns in an unsupervised manner by trying to predict the future (read Hawkins’ ‘On Intelligence’, several researchers have posited this), and novelty and our notion of surprise is when something happens that we’d didn’t predict or come close to expecting. It certainly seems our brains place a premium on this sort of experience, TV shows, movies and books that are predictable are a lot less enjoyable than those that continually defy our expectations (Breaking Bad anyone?). More formally, detecting novelty is referred to as ‘outlier detection’ in machine learning and statistics, at least certain forms of novelty would come under this area of research. A lot of work has gone into this area, as it’s used to detect things like fraudulent transactions and distinguishing between human and bot clicks in online adverts (for c.p.c. models).


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s