April, 2022: Hi there! Thanks for visiting my personal webpage. Unfortunately, the current page is a bit outdated. New content is coming very soon, so please visit again. Here's my CV in the meantime. :) I'm Shashank, a Masters student in the Computer Science Department at the University of Illinois at Urbana-Champaign, where I'm working with Prof. Dan Roth as a part of his amazing CogComp research group! In general, I work on the intersection of Machine Learning and Natural Language Processing, however, I'm quite excited about Guided Language Generation tasks (like Dialogue systems, Abstractive Summarization, Question Answering), and learning generic Sentence and Document Embeddings. On the ML side, I am interested in studying Deep Generative and Deep Structured models, and like to think about how to Distribute training and inference, and interpret ML models in general. Currently, my work centers around Zero-Shot Learning for Text Classification, and learning and evaluating useful Document Embeddings. Prior to coming to UIUC, I gained plenty of research experience through several research assistantships, and had the pleasure of being a part of some great research labs. For instance, I spent 9 amazing months working with Prof. Gerhard Weikum and Prof. Denilson Barbosa on Knowledge-Base Construction at the Max Planck Institute for Informatics. I also had the great pleasure of working closely with Prof. Soumen Chakrabarti on Scalable Entity Search and Disambiguation System at IIT-Bombay, where I was a Research Assistant for about 1.5 years. I was also fortunate to be a part of the amazing Ad-Prediction team at Yahoo Labs, where I worked on some challenging problems in Search and Display Ad-platforms, under the guidance of Prof. Sanjay Chawla and Prof. Shivaram Kalyanakrishnan. Before my foray into serious research, I spent some wonderful years at BITS - Pilani, from where I obtained my Bachelor's degree in Computer Science. I spent most of my time there working on my idea of Client-Side Search Personalization.

News

---

• 02/2018: Will be joining the Microsoft's A.I. for Office team as a Data Scientist in April, 2018.
• 06/2017: Gave a tutorial on Modular Networks in our Deep Learning Reading Group.
• 05/2017: Started organizing the Deep Learning Discussion Group at UIUC. Checkout our schedules (Fall'17, Summer'17) and reading list.
• 02/2017: Gave a tutorial on GANs in the Deep Learning Special Topics course. [PDF] [Slides]
• 12/2016: Had an amazing time TA-ing an Introductory course on Machine Learning (CS446).
• 08/2015: Started school at the University of Illinois at Urbana Champaign as a Masters student; will be guided by Prof. Dan Roth.
• 12/2013: Our paper on Web-Scale Entity Annotation using MapReduce was published at HiPC, 2013. [PDF]
• 07/2013: Invited Talk on Web-Scale Entity Annotation using MapReduce at the Yahoo Summer School on Information Retrieval & the Semantic Web. [Slides]
• 06/2013: Presented a poster on Web-Scale Entity Disambiguation at the NetApp University Day. [PDF]

Publications

---

• Shashank Gupta, Varun Chandramouli, Soumen Chakrabarti, "Web-scale Entity Annotation Using MapReduce." HiPC 2013. [PDF] [Slides]

Projects

---

Unsupervised Text Classification

Most existing Text Classification techniques are supervised in nature, and thus require the end-user to provide supervision for every topic/concept of interest. However, getting ample supervision might not always be possible. Moreover, the supervised learning techniques just treat the target label as an ID in their system, and don't really exploit the semantic meaning of the target label. With the advent of large-scale Knowledge-Bases (KBs), it should be possible now to bypass explicit supervision by exploiting the information in KBs.

Towards this goal, we are building up on our previous work on Dataless Classification, and have experimented with new ESA-style entity embeddings. We also modified the Word2Vec objective to introduce an explicit notion of topical similarity between words. Realizing the importance of compositionality, we are now in the process of learning topic-sensitive document embeddings. Our initial study on existing architectures for text classification demonstrated the importance of using a hierarhical model with hierarchical attention.



One-Shot Topic Classification (May, 2017 - Present)[Code ... Coming Soon]

Aim is to use Wikipedia to learn document representations that can enable one-shot learning of topics.
• Created a new text classification dataset from Wikipedia where categories are the labels.
• Densified Wikipedia category graph to tackle sparsity issues in the dataset.
• Currently, in the process of trying out several neural models on this dataset.
• In the near future, this trained network will be used for producing document representations, which will then be used alongside a meta-learning algorithm for one-shot topic classification on arbitrary datasets.



Injecting Topics into Word Embeddings for Unsupervised Classification (January – May, 2017)[Code ... Coming Soon]

Aim was to somehow inject a notion of topical similarity into the word embeddings
• Augmented the Word2Vec loss with an additional Topical loss.
• Used Wikipedia’s categorization to provide the topical similarity signal.
• Used these new embeddings for unsupervised document classification.



A Study of Neural Models for Text Classification (August – December, 2016)[Code ... Coming Soon]

Aim was to benchmark several neural models for various document classification tasks (topics, sentiment)
• Coded up the hierarchical attention model for document classification.
• Carried out an ablation study by removing several components from the model.
• Both Hierarchical RNN and Attention components were found to be vital for good performance.



Unsupervised Document Classification using Semantic Embeddings (August, 2015 – July, 2016)

Aim was to automatically label a document collection with user-provided topics without any supervision.
• Key idea was to represent Topics and Documents with Semantic Embeddings and use nearest-neighbor for classification.
• Created ESA-style sparse embeddings for Entities.
• Used composition of Entity and Word embeddings to represent documents.
• Entity embeddings showed promise in entity similarity tasks.

Text Generation

Conditional Text Generation (January – May, 2017)[Code ... Coming Soon]

Aim was to try out Deep Generative Models for text generation.
• Tried out GANs and VAEs for sentiment-conditioned movie review generation.
• Tried both Policy-Gradient and Gumbel-Softmax to make GANs handle discrete outputs.
• Used Curriculum Learning and a conditional language model to bootstrap the GANs.
• Found both GANs and VAEs difficult to train.
• Ultimately, a simple conditional language model was found to have the best performance.

Joint Learning

Joint NER, RelEx and CoRef using CCMs (January – May, 2016)[Code]

Aim was to develop a joint model for NER, Relation Extraction and Coreference Resolution
• Experimented with various off-the-shelf tools to note poor performance.
• Trained classifiers for each of the tasks using CogComp-SL and CogComp-NLP
• Developed a pipelined system to couple all the classifiers.

Knowledge-Base Construction

Agile NERD for KB-Lifecycle (August, 2014 – April, 2015)

Aim was to develop a single system capable of Entity Disambiguation (NERD) and Automatic KB Construction.
• Promise being two-fold – automated construction of a real-time canonicalized KB, along with an iterative annotation of emerging entities on corpus.
• Worked on differentiating mentions of emerging entities from mentions where NERD should abstain from linking due to insufficient confidence.
• Used the disagreement between an ensemble of annotators to signal abstention on a given mention.
• Preliminary experiments showed promise in identifying mentions worthy of abstention.

Named Entity Disambiguation

Effect of Annotation Quality on Entity Ranking Accuracy (June, 2013 – June, 2014)

Developed a scalable NERD system that can annotate entities with calibrated confidence scores
• Aim was to annotate entities on a web-scale corpus with calibrated scores, so as to be able to design robust entity ranking techniques.
• Developed a distributed load-balanced framework for training millions of Logistic Regression models.
• Also developed scalable solutions for storage and retrieval of these models, so as to be able to quickly annotate and stream through a web-scale corpus.



NERD Annotation Accuracy Improvement (October, 2013 – June, 2014)

Re-designed NERD system to improve F1 by ~30%, and Recall by ~100%, without affecting scalability
• Improved Recall by optimizing Wikipedia parsing to extract more training data.
• Benchmarked our system against other annotators on standard datasets.
• Engineered features to improve the F1 score.



Web-Scale Entity Annotation Using MapReduce (January – May, 2013)

Designed a scalable entity annotation and indexing framework in Hadoop.
• Developed a MapReduce implementation for web-scale Entity Disambiguation.
• Designed custom-key partitioning strategies to mitigate the load-skew problem of a simple MapReduce implementation.
• Improved performance (time to completion) by over 5.4 times.
• Published this work in High Performance Computing (HiPC), 2013.
• This framework enabled us to annotate and index a web-scale corpus in a reasonable amount of time.

Ad Sciences

User Response Prediction for Non-Guaranteed Display Ad Delivery (June – December, 2012)

Improved the accuracy of the user-click prediction model.
• Analyzed the entire Display Advertising Exchange platform (Right Media Exchange).
• Exploited domain expertise for mining new features.
• Crunched Petabytes of data to analyze coverage and signal of each feature.
• Showed promise for the approach of training different models for different partitions of training data, by identifying one promising data-partitioning strategy.



CampOps – Automated Campaign Optimization for Search Advertising (January – June, 2012)

Prototyped a tool to automate advertiser’s accounts’ optimization
• Analyzed the entire Sponsored Search Analytics Platform and noticed serious bottlenecks.
• As a result, prototyped a tool that:
  • predicted number of Impressions, Clicks, Conversions and the Amount Charged for each combination of targeting attributes by learning from competitors using machine learning techniques.
  • determined competitors algorithmically by forming a relationship graph between search terms and running community detection algorithms to cluster them, where each cluster roughly maps to one product in the market.
  • provided better keyword suggestions to the advertisers.
  • converted the entire problem of automating advertiser’s accounts’ optimization into a mere Resource Allocation Problem.
  • was expected to generate huge amount of revenue for the company and better ROI for advertisers.
  • was capable of automating and improving the entire work carried out by a dedicated team of Analysts.

Client-side Search Personalization

User Profiling on the Client-side (August, 2010 – December, 2011)

Prototyped a browser extension to model user intention and re-rank search results on the client-side.
• Extension recorded client-side exclusive browsing features like tab-switching, browser minimization, text-selection, downloads, number of open tabs, depth/breadth-vise click chain.
• Used Neural Networks along with Google Site Search for the supervised learning of the relation of these features with the interest of a user in a webpage.
• Created user profile from the pages thus determined to be relevant.
• Then used this profile to personalize search results using User Interest Hierarchy (UIH).
• Overall, motivated the usefulness of client-side browsing features in personalization of search results.

Miscellaneous

Online Comprehensive Examination Software (May – July, 2010)

Developed a J2EE software for subjective online examinations.
• Aim was to minimize the wastage of paper in carrying out the subjective examinations.
• Through this software, the entire academics of the institution (including exams) was made online.

CV

---

Here's my CV (Last updated: April, 2022).

Contact

---

• Email: shashank.nlp ¯\_(ツ)_/¯ gmail.com
• Github, Twitter, Linkedin

Resources

---

I spend countless hours following fellow researchers on social media, and have learnt a great deal from their blogs, mailing lists, reading lists, tweets, podcasts, courses etc. Following is a feeble attempt at showing my gratitude and paying it forward by compiling a list of some useful resources.

Blogs

• OpenAI's blog
• Karpathy's blog
• David Ha's awesome blog.
• Chris Olah's blog
• Distill.pub
• UC Berkeley AI group's blog
• Denny Britz's blog
• Arthur's awesome series on Reinforcement Learning.
• Sebastian's NLP blog
• Prof. Sanjeev Arora's blog
• Prof. Hal Daume's NLP blog

Newsletters

• Jack's Import AI Newsletter
• Denny's "The wild week in AI"
• Sebastian's NLP News

Twitter

A lot of NLP and Deep Learning researchers are on Twitter, and very actively discuss new and interesting papers. Thanks to Prof. Jason Baldridge, we now have a nearly exhaustive list of them.

Podcasts

• AI2's NLP Highlights
• Talking Machines

Courses

• Coming soon ... 🙂