Visualizing recurrent neural networks: a funny talk at RE.WORK 2016 San Francisco

Andrej Karpathy is a PhD student (maybe completed PhD…) from Stanford University. He is involved with a PhD about deep neural networks, specifically convolutional neural networks and recurrent neural networks. He also joined the OpenAI research initiative as one of its prominent researchers. He delivered a nice, relaxed and funny talk at last year’s RE.WORK 2016 event in San Francisco. The video of talk is below and The Intelligence of Information wanted to share it with followers.

The video features some of the research that Andrej is involved with, the corresponding papers and links, and some GitHub repositories that he pulls through. The interested reader is well advised to pause the video and Google the respective papers and links, even if the research is somewhat outdated already. There is always some value in revisiting some of these papers. Andrej is also a lecturer in the convolutional neural network for visual recognition’s Stanford CS231n class and the resources are presented in the last slide.

One important take away from the talk is the part about sequential recurrent neural networks applied in the context of translation and text processing. The current deep neural networks models are still limited in the interpretability of their black-box characteristics, so work on lines of research on improving interpretability is welcomed. The details that Andrej disclose about his and fellow’s efforts on this aspect of deep neural networks research  seems to point a possible direction for a line of further research. Indeed this part of the talk was the funny part, with the laugher by Andrej and the audience being as rewarding as the some of the Shakespeare sonnets or the mathematical theorems that the recurrent neural network is able to automatically reproduce (… and the cooking recipes or the twitter bots…). 😊

The final word goes for the connection between computer vision algorithms, the dense semantic captioning that the combination of convolutional neural networks and recurrent neural networks Andrej used in one of papers presented is able to accomplish, which again is another vindication of the view the these artificial neural networks really perform computations similar to the ones performed by the human brain (this similarity we perceive must somehow tell something about both artificial and biological neural networks as to how they function, even if we currently do not fully understand completely how they both function).

featured image: Andrej Karpathy, Research Scientist, OpenAI – RE•WORK Deep Learning Summit 2016 #reworkDL


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s