Over the past year, perhaps the most cited paper across the software industry is Attention is All You Need that is at the heart of ChatGPT and GPT transformer models. The first thing you will notice in the paper is the Attention formula: $$\text{Attention(Q, K, V)} = \text{softmax}(\frac{QK^T}{\sqrt{d_k}})V$$ Unfortunately, very few sources have delved into … Continue reading The Core of Attention is Communication
Category: Machine Learning
Rust and Node.js: Harmonizing Performance and Safety
Prelude In the Rust world, the interaction between Python and Rust is very well-known through the amazing PyO3 ecosystem. There is a similar relation between Python and Javascript in particular Node.js that I'm going to describe in this post. All the code is available here. Most programming language interactions happen through C layer ABI i.e. … Continue reading Rust and Node.js: Harmonizing Performance and Safety
Notes on the Current State of LLM Frameworks
This post tries to shed some light on the rapidly changing LLM frameworks in particular, LangChain (LC) and Llama-index (LI). Library vs. Framework It's tricky to draw a clear boundary between a package/library and a framework, but for the sake of discussion, let's look at some well-known examples Packages: Numpy falls into this category. It … Continue reading Notes on the Current State of LLM Frameworks
The State of Machine Learning in Rust
Every once in a while this topic comes up on a social media or Rust user channel. I'd like to describe briefly the way I see where things are going by a little bit of history as well as some information about existing flux of Machine Learning/Deep Learning frameworks and major recent trends. Brief history … Continue reading The State of Machine Learning in Rust
NIPS, AI hype and the lost rigor
Warning: This post contains a mixture of excitements, frustrations and rants! Today, Machine Learning/Deep Learning people have been sharing their great excitements over Ali Rahimi's talk at NIPS (from min 57 onwards). Undoubtedly, it's a great talk and you should check it out if you care about fundamental issues and the lost rigor in Deep Learning … Continue reading NIPS, AI hype and the lost rigor
What’s up with word embedding?
Word embedding is one of the interesting areas of research in Natural Language Processing. There are huge amount of materials with a lot of interesting ideas. I have been studying some of them lately and in this post, I'd like to create a brief account of the ideas I have found most interesting so far. … Continue reading What’s up with word embedding?
From Machine Learning to Formal Math and Type Theory
The idea of this post was sparkled from the new paper Developing Bug-Free Machine Learning Systems with Formal Mathematics. Meanwhile, I have had the idea of writing about what you're going to read for a long time and this paper happily forced me to do it finally! The first and final parts are about my journey and … Continue reading From Machine Learning to Formal Math and Type Theory
Restaurant Revenue Prediction with BART Machine
In this post, I'd like to show you how to use the newly written package on Bayesian Additive Regression Trees i.e. BART Machine for Restaurant Revenue Prediction with R. The datasets are part of the passed Kaggle competition that can be found here. What is BART? BART is the Bayesian sibling of Random Forest. For … Continue reading Restaurant Revenue Prediction with BART Machine