The Core of Attention is Communication

Over the past year, perhaps the most cited paper across the software industry is Attention is All You Need that is at the heart of ChatGPT and GPT transformer models. The first thing you will notice in the paper is the Attention formula: $$\text{Attention(Q, K, V)} = \text{softmax}(\frac{QK^T}{\sqrt{d_k}})V$$ Unfortunately, very few sources have delved into … Continue reading The Core of Attention is Communication

Rust and Node.js: Harmonizing Performance and Safety

Prelude In the Rust world, the interaction between Python and Rust is very well-known through the amazing PyO3 ecosystem. There is a similar relation between Python and Javascript in particular Node.js that I'm going to describe in this post. All the code is available here. Most programming language interactions happen through C layer ABI i.e. … Continue reading Rust and Node.js: Harmonizing Performance and Safety

Notes on the Current State of LLM Frameworks

This post tries to shed some light on the rapidly changing LLM frameworks in particular, LangChain (LC) and Llama-index (LI). Library vs. Framework It's tricky to draw a clear boundary between a package/library and a framework, but for the sake of discussion, let's look at some well-known examples Packages: Numpy falls into this category. It … Continue reading Notes on the Current State of LLM Frameworks

Announcement 📢 Releasing dlpackrs

DLPack is the standard in-memory data format that facilitates zero-cost tensor transfer across major Deep Learning frameworks (PyTorch, TensorFlow and TVM) and the supported Python Array processing frameworks such as Numpy, CuPy. The dlpackrs provides a safe idiomatic Rust binding where Rust ndarray and tensor frameworks can use it to gain the same kind of … Continue reading Announcement 📢 Releasing dlpackrs

Announcement 📢 Create your own programming language with Rust

After almost a year from my last blog post, in this short post I'm very happy to announce that I'm writing a free online book where early chapters are available now. I've explained my motivations and goals in the introduction. The accompanying codes are also available on my GitHub. Feedbacks are welcome and happy learning. … Continue reading Announcement 📢 Create your own programming language with Rust

Rust std study series: Interior mutability

Continuing the standard library study, it's time for Cell<T>! Rust compiler enforces multiple reads access and a single write access mutually exclusive, i.e. either multiple shared references & or one and only one mutable reference & mut. So essentially, Rust prevents the evil of aliasing and mutation between multiple threads. Cell<T> is a sharable mutable … Continue reading Rust std study series: Interior mutability

Rust std study series: LinkedList

Continuing from Rust standard library study series, it's time for LinkedList<T>. Note that implementation are taken from Rust stable v1.33.0. A doubly-linked list with owned nodes. The LinkedList allows pushing and popping elements at either end in constant time. Almost always it is better to use Vec or VecDeque instead of LinkedList. In general, array-based … Continue reading Rust std study series: LinkedList

From Machine Learning to Formal Math and Type Theory

The idea of this post was sparkled from the new paper Developing Bug-Free Machine Learning Systems with Formal Mathematics. Meanwhile, I have had the idea of writing about what you're going to read for a long time and this paper happily forced me to do it finally! The first and final parts are about my journey and … Continue reading From Machine Learning to Formal Math and Type Theory