Cross Entropy Loss Connection to GPT Models
Cross-entropy loss isn’t a heuristic, it is maximum likelihood estimation with a sign flip. It also shows how the same math powers GPT training.
Posts by Tag
- llm 7
- python 5
- machine learning 3
- scikit-learn 2
- deep-learning 2
- r 1
- superml 1
- text matching 1
- sqlite 1
- asyncio 1
- parallel 1
- vllm 1
- gpu 1
- pytorch 1
- transformers 1
- rope 1
- learning-to-rank 1
- lightgbm 1
- fine-tuning 1
- deeplearning 1
llm
RLHF vs RLAIF vs RLVR: The Three Ways to Teach AI Models
Understanding the basics of RLHF vs RLAIF vs RLVR for AI feedback comparison
Complete Guide to SFT and DPO Fine-tuning with Axolotl
Master the complete pipeline for fine-tuning Large Language Models using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with Axolotl f...
Simple Guide to RoPE Scaling in Large Language Models
Understanding RoPE Scaling and how it enables LLMs to handle longer contexts
Basics of Gradient Accumulation and Checkpointing to train LLMs
Basics of gradient accumulation and gradient checkpointing to train LLMs
Simple Guide on FLOPs in GPU Utilisation
Understanding basics of flops and how they influence gpu computation
5 Lessons from Deploying LLMs in Production using vLLM
Lessons learned from deploying large language models in productions using vLLM
python
Hands on tutorial with Polars DataFrames in Python
Tutorial to work on large datasets in memory Polars DataFrames in Python
Practical Tutorial on Asyncio in Python 3.7
Understanding asyncio using simple examples in python 3.7
Useful Full Text Search (FTS) queries with sqlite in Python 3.7
Learning to use FTS5 with sqlite for full text search applications
Packaging a Machine Learning Project using Python PEX
Use pex python library to package a python project for deployment.
Banco Data Model
this line will show up as preview on the posts page
machine learning
How to create TF-IDF matrix using ngrams in R?
Create tfidf matrix in R just like using scikit-learn
Packaging a Machine Learning Project using Python PEX
Use pex python library to package a python project for deployment.
Banco Data Model
this line will show up as preview on the posts page
scikit-learn
Packaging a Machine Learning Project using Python PEX
Use pex python library to package a python project for deployment.
Banco Data Model
this line will show up as preview on the posts page
deep-learning
Complete Guide to SFT and DPO Fine-tuning with Axolotl
Master the complete pipeline for fine-tuning Large Language Models using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with Axolotl f...
Basics of Gradient Accumulation and Checkpointing to train LLMs
Basics of gradient accumulation and gradient checkpointing to train LLMs
r
How to create TF-IDF matrix using ngrams in R?
Create tfidf matrix in R just like using scikit-learn
superml
How to create TF-IDF matrix using ngrams in R?
Create tfidf matrix in R just like using scikit-learn
text matching
Useful Full Text Search (FTS) queries with sqlite in Python 3.7
Learning to use FTS5 with sqlite for full text search applications
sqlite
Useful Full Text Search (FTS) queries with sqlite in Python 3.7
Learning to use FTS5 with sqlite for full text search applications
asyncio
Practical Tutorial on Asyncio in Python 3.7
Understanding asyncio using simple examples in python 3.7
parallel
Practical Tutorial on Asyncio in Python 3.7
Understanding asyncio using simple examples in python 3.7
vllm
5 Lessons from Deploying LLMs in Production using vLLM
Lessons learned from deploying large language models in productions using vLLM
gpu
Simple Guide on FLOPs in GPU Utilisation
Understanding basics of flops and how they influence gpu computation
pytorch
Basics of Gradient Accumulation and Checkpointing to train LLMs
Basics of gradient accumulation and gradient checkpointing to train LLMs
transformers
Simple Guide to RoPE Scaling in Large Language Models
Understanding RoPE Scaling and how it enables LLMs to handle longer contexts
rope
Simple Guide to RoPE Scaling in Large Language Models
Understanding RoPE Scaling and how it enables LLMs to handle longer contexts
learning-to-rank
How to Build Multi-Objective Ranking Models with LightGBM
Learning to rank with lambdarank multi objective pairwise ranking models using lightgbm
lightgbm
How to Build Multi-Objective Ranking Models with LightGBM
Learning to rank with lambdarank multi objective pairwise ranking models using lightgbm
fine-tuning
Complete Guide to SFT and DPO Fine-tuning with Axolotl
Master the complete pipeline for fine-tuning Large Language Models using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with Axolotl f...
deeplearning
Cross Entropy Loss Connection to GPT Models
Cross-entropy loss isn’t a heuristic, it is maximum likelihood estimation with a sign flip. It also shows how the same math powers GPT training.