痛点分析发布于 2026/05/31
痛点为 AI 基于上游原始证据的初步提炼；未包含额外中国市场检索。
痛点
用户想训练自己的大语言模型（LLM），但现有流程从数据下载到文本生成缺乏清晰、直接的指导，导致入门门槛高、试错成本大。GitHub 仓库描述强调“直接了当的方法”，暗示用户常因分散的教程、复杂的框架配置和缺乏端到端示例而卡在中间步骤，无法快速验证模型效果。这种摩擦造成时间浪费和挫败感，尤其对希望快速实验的开发者而言，重复搜索和调试会显著拖慢项目进度。
README
README summary
<div align="center" Train LLM From Scratch I am Looking for a PhD position in AI . GitHub
Repository
Repository metadata
- Owner
- FareedKhan-dev
- License
- MIT
- Default branch
- main
- Days since created
- 504
- Days since last push
- 9
Signals
Repository signals
- Watchers
- 2,892
- Open issues
- 5
Topics
geminilarge-language-modelsllmopenaitrainingtransformers
Contributors
Contributor snapshot
- Contributor count
- 6
- Top contributor share
- 0.489
- Top contributors
- FareedKhan-dev, TianyiQ, eltociear, Jah-yee, its-not-rocket-science
源数据· Raw Archive
- source
- GitHub Trending
- upstream_source
- github_trending
- upstream_item_id
- FareedKhan-dev--train-llm-from-scratch
- daily_ranking_item_id
- 84018286-af11-4287-b333-0cc5add44f7d
- rank_date
- 2026-06-01
- rank
- 9
- name
- train-llm-from-scratch
- tagline
- A straightforward method for training your LLM, from downloading data to generating text.
- description
- A straightforward method for training your LLM, from downloading data to generating text.
- votes_count
- 2,892
- thumbnail_url
- https://github.com/FareedKhan-dev.png
- og_image_url
- https://github.com/FareedKhan-dev.png
topics
geminilarge-language-modelsllmopenaitrainingtransformers
media / source-specific data
{
"topics": [
"gemini",
"large-language-models",
"llm",
"openai",
"training",
"transformers"
],
"is_fork": false,
"license": "MIT",
"language": "Jupyter Notebook",
"owner_type": "User",
"forks_total": 437,
"has_funding": false,
"is_archived": false,
"owner_login": "FareedKhan-dev",
"stars_today": 627,
"stars_total": 2892,
"homepage_url": "https://medium.com/@fareedkhandev/building-a-2-billion-parameter-llm-from-scratch-using-python-1325cb05d6fb",
"default_branch": "main",
"last_pushed_at": "2026-05-22T04:17:43Z",
"readme_summary": "<div align=\"center\" Train LLM From Scratch I am Looking for a PhD position in AI . GitHub",
"repo_full_name": "FareedKhan-dev/train-llm-from-scratch",
"watchers_count": 2892,
"last_updated_at": "2026-05-31T21:58:39Z",
"top_contributors": [
{
"login": "FareedKhan-dev",
"contributions": 22
},
{
"login": "TianyiQ",
"contributions": 19
},
{
"login": "eltociear",
"contributions": 1
},
{
"login": "Jah-yee",
"contributions": 1
},
{
"login": "its-not-rocket-science",
"contributions": 1
}
],
"contributor_count": 6,
"funding_platforms": [],
"open_issues_count": 5,
"days_since_created": 504,
"created_at_on_source": "2025-01-12T04:03:05Z",
"days_since_last_push": 9,
"top_contributor_share": 0.489
}raw_payload
{
"fetched_at": "2026-05-31T22:00:31.891Z",
"trending_repo": {
"url": "https://github.com/FareedKhan-dev/train-llm-from-scratch",
"name": "train-llm-from-scratch",
"rank": 9,
"forks": 437,
"owner": "FareedKhan-dev",
"stars": 2892,
"fullName": "FareedKhan-dev/train-llm-from-scratch",
"language": "Jupyter Notebook",
"avatarUrl": "https://github.com/FareedKhan-dev.png",
"rawSummary": "<div class=\"float-right d-flex\">\n\n <div data-view-component=\"true\" class=\"BtnGroup d-flex\">\n <a href=\"/login?return_to=%2FFareedKhan-dev%2Ftrain-llm-from-scratch\" rel=\"nofollow\" data-hydro-click=\"{"event_type":"authentication.click","payload":{"location_in_page":"star button","repository_id":915520679,"auth_type":"LOG_IN","originating_url":"https://github.com/trending?since=daily","user_id":null}}\" data-hydro-click-hmac=\"fc641fcf74a0fa0145d47086fd8b888057ba50a91842684c8420119113864404\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-sw btn-sm btn\"> <svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star v-align-text-bottom d-none d-md-inline-block mr-2 tmp-mr-2\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg><svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star mr-0 tmp-mr-0 v-align-text-bottom d-inline-block d-md-none\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n <span data-view-component=\"true\" class=\"d-none d-md-inline\">\n Star\n</span>\n</a></div>\n </div>\n\n <h2 class=\"h3 lh-condensed\">\n <a data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"REPOSITORY","click_visual_representation":"REPOSITORY_NAME_HEADING","actor_id":null,"record_id":915520679,"originating_url":"https://github.com/trending?since=daily","user_id":null}}\" data-hydro-click-hmac=\"18d2c5d41c0758f4d90177381abe3e05b465ce54e8772a6b376a683f8619444a\" href=\"/FareedKhan-dev/train-llm-from-scratch\" data-view-component=\"true\" class=\"Link\"><svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted\">\n <path d=\"M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z\"></path>\n</svg>\n\n <span data-view-component=\"true\" class=\"text-normal\">\n FareedKhan-dev /\n</span>\n train-llm-from-scratch</a> </h2>\n\n <p class=\"col-9 color-fg-muted my-1 tmp-pr-4\">\n A straightforward method for training your LLM, from downloading data to generating text.\n </p>\n\n <div class=\"f6 color-fg-muted mt-2\">\n <span class=\"tmp-mr-3 d-inline-block ml-0 tmp-ml-0\">\n <span class=\"repo-language-color\" style=\"background-color: #DA5B0B\"></span>\n <span itemprop=\"programmingLanguage\">Jupyter Notebook</span>\n</span>\n\n\n <a href=\"/FareedKhan-dev/train-llm-from-scratch/stargazers\" data-view-component=\"true\" class=\"tmp-mr-3 Link Link--muted d-inline-block\"><svg aria-label=\"star\" role=\"img\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n 2,892</a>\n <a href=\"/FareedKhan-dev/train-llm-from-scratch/forks\" data-view-component=\"true\" class=\"tmp-mr-3 Link Link--muted d-inline-block\"><svg aria-label=\"fork\" role=\"img\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-repo-forked\">\n <path d=\"M5 5.372v.878c0 .414.336.75.75.75h4.5a.75.75 0 0 0 .75-.75v-.878a2.25 2.25 0 1 1 1.5 0v.878a2.25 2.25 0 0 1-2.25 2.25h-1.5v2.128a2.251 2.251 0 1 1-1.5 0V8.5h-1.5A2.25 2.25 0 0 1 3.5 6.25v-.878a2.25 2.25 0 1 1 1.5 0ZM5 3.25a.75.75 0 1 0-1.5 0 .75.75 0 0 0 1.5 0Zm6.75.75a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5Zm-3 8.75a.75.75 0 1 0-1.5 0 .75.75 0 0 0 1.5 0Z\"></path>\n</svg>\n 437</a>\n <span data-view-component=\"true\" class=\"tmp-mr-3 d-inline-block\">\n Built by\n\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/FareedKhan-dev/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/FareedKhan-dev\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/63067900?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@FareedKhan-dev\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/TianyiQ/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/TianyiQ\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/34389237?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@TianyiQ\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/konglingfan/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/konglingfan\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/8125957?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@konglingfan\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/eltociear/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/eltociear\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/22633385?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@eltociear\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/its-not-rocket-science/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/its-not-rocket-science\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/87320439?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@its-not-rocket-science\" /></a>\n</span>\n <span data-view-component=\"true\" class=\"d-inline-block float-sm-right\">\n <svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n 627 stars today\n</span> </div>",
"starsToday": 627,
"description": "A straightforward method for training your LLM, from downloading data to generating text."
},
"snapshot_version": "github_trending_v2"
}source_raw_snapshot
{
"id": "20a6832f-53ba-4ac1-a926-0a5c3c3be23c",
"daily_ranking_item_id": "84018286-af11-4287-b333-0cc5add44f7d",
"source": "github_trending",
"external_id": "FareedKhan-dev--train-llm-from-scratch",
"fetched_at": "2026-05-31T22:00:31.891Z",
"trending_page_raw": {
"url": "https://github.com/FareedKhan-dev/train-llm-from-scratch",
"name": "train-llm-from-scratch",
"rank": 9,
"forks": 437,
"owner": "FareedKhan-dev",
"stars": 2892,
"language": "Jupyter Notebook",
"full_name": "FareedKhan-dev/train-llm-from-scratch",
"avatar_url": "https://github.com/FareedKhan-dev.png",
"description": "A straightforward method for training your LLM, from downloading data to generating text.",
"raw_summary": "<div class=\"float-right d-flex\">\n\n <div data-view-component=\"true\" class=\"BtnGroup d-flex\">\n <a href=\"/login?return_to=%2FFareedKhan-dev%2Ftrain-llm-from-scratch\" rel=\"nofollow\" data-hydro-click=\"{"event_type":"authentication.click","payload":{"location_in_page":"star button","repository_id":915520679,"auth_type":"LOG_IN","originating_url":"https://github.com/trending?since=daily","user_id":null}}\" data-hydro-click-hmac=\"fc641fcf74a0fa0145d47086fd8b888057ba50a91842684c8420119113864404\" aria-label=\"You must be signed in to star a repository\" data-view-component=\"true\" class=\"tooltipped tooltipped-sw btn-sm btn\"> <svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star v-align-text-bottom d-none d-md-inline-block mr-2 tmp-mr-2\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg><svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star mr-0 tmp-mr-0 v-align-text-bottom d-inline-block d-md-none\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n <span data-view-component=\"true\" class=\"d-none d-md-inline\">\n Star\n</span>\n</a></div>\n </div>\n\n <h2 class=\"h3 lh-condensed\">\n <a data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"REPOSITORY","click_visual_representation":"REPOSITORY_NAME_HEADING","actor_id":null,"record_id":915520679,"originating_url":"https://github.com/trending?since=daily","user_id":null}}\" data-hydro-click-hmac=\"18d2c5d41c0758f4d90177381abe3e05b465ce54e8772a6b376a683f8619444a\" href=\"/FareedKhan-dev/train-llm-from-scratch\" data-view-component=\"true\" class=\"Link\"><svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-repo mr-1 tmp-mr-1 color-fg-muted\">\n <path d=\"M2 2.5A2.5 2.5 0 0 1 4.5 0h8.75a.75.75 0 0 1 .75.75v12.5a.75.75 0 0 1-.75.75h-2.5a.75.75 0 0 1 0-1.5h1.75v-2h-8a1 1 0 0 0-.714 1.7.75.75 0 1 1-1.072 1.05A2.495 2.495 0 0 1 2 11.5Zm10.5-1h-8a1 1 0 0 0-1 1v6.708A2.486 2.486 0 0 1 4.5 9h8ZM5 12.25a.25.25 0 0 1 .25-.25h3.5a.25.25 0 0 1 .25.25v3.25a.25.25 0 0 1-.4.2l-1.45-1.087a.249.249 0 0 0-.3 0L5.4 15.7a.25.25 0 0 1-.4-.2Z\"></path>\n</svg>\n\n <span data-view-component=\"true\" class=\"text-normal\">\n FareedKhan-dev /\n</span>\n train-llm-from-scratch</a> </h2>\n\n <p class=\"col-9 color-fg-muted my-1 tmp-pr-4\">\n A straightforward method for training your LLM, from downloading data to generating text.\n </p>\n\n <div class=\"f6 color-fg-muted mt-2\">\n <span class=\"tmp-mr-3 d-inline-block ml-0 tmp-ml-0\">\n <span class=\"repo-language-color\" style=\"background-color: #DA5B0B\"></span>\n <span itemprop=\"programmingLanguage\">Jupyter Notebook</span>\n</span>\n\n\n <a href=\"/FareedKhan-dev/train-llm-from-scratch/stargazers\" data-view-component=\"true\" class=\"tmp-mr-3 Link Link--muted d-inline-block\"><svg aria-label=\"star\" role=\"img\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n 2,892</a>\n <a href=\"/FareedKhan-dev/train-llm-from-scratch/forks\" data-view-component=\"true\" class=\"tmp-mr-3 Link Link--muted d-inline-block\"><svg aria-label=\"fork\" role=\"img\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-repo-forked\">\n <path d=\"M5 5.372v.878c0 .414.336.75.75.75h4.5a.75.75 0 0 0 .75-.75v-.878a2.25 2.25 0 1 1 1.5 0v.878a2.25 2.25 0 0 1-2.25 2.25h-1.5v2.128a2.251 2.251 0 1 1-1.5 0V8.5h-1.5A2.25 2.25 0 0 1 3.5 6.25v-.878a2.25 2.25 0 1 1 1.5 0ZM5 3.25a.75.75 0 1 0-1.5 0 .75.75 0 0 0 1.5 0Zm6.75.75a.75.75 0 1 0 0-1.5.75.75 0 0 0 0 1.5Zm-3 8.75a.75.75 0 1 0-1.5 0 .75.75 0 0 0 1.5 0Z\"></path>\n</svg>\n 437</a>\n <span data-view-component=\"true\" class=\"tmp-mr-3 d-inline-block\">\n Built by\n\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/FareedKhan-dev/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/FareedKhan-dev\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/63067900?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@FareedKhan-dev\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/TianyiQ/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/TianyiQ\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/34389237?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@TianyiQ\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/konglingfan/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/konglingfan\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/8125957?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@konglingfan\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/eltociear/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/eltociear\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/22633385?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@eltociear\" /></a>\n <a class=\"d-inline-block\" data-hydro-click=\"{"event_type":"explore.click","payload":{"click_context":"TRENDING_REPOSITORIES_PAGE","click_target":"CONTRIBUTING_DEVELOPER","click_visual_representation":"DEVELOPER_AVATAR","actor_id":null,"record_id":null,"originating_url":"https://github.com/trending","user_id":null}}\" data-hydro-click-hmac=\"14a81dd1b5cd12f2e95030cb17362cb06eaa0349997a1d520f0bec614d40dc93\" data-hovercard-type=\"user\" data-hovercard-url=\"/users/its-not-rocket-science/hovercard\" data-octo-click=\"hovercard-link-click\" data-octo-dimensions=\"link_type:self\" href=\"/its-not-rocket-science\"><img class=\"avatar mb-1 avatar-user\" src=\"https://avatars.githubusercontent.com/u/87320439?s=40&v=4\" width=\"20\" height=\"20\" alt=\"@its-not-rocket-science\" /></a>\n</span>\n <span data-view-component=\"true\" class=\"d-inline-block float-sm-right\">\n <svg aria-hidden=\"true\" data-component=\"Octicon\" height=\"16\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" data-view-component=\"true\" class=\"octicon octicon-star\">\n <path d=\"M8 .25a.75.75 0 0 1 .673.418l1.882 3.815 4.21.612a.75.75 0 0 1 .416 1.279l-3.046 2.97.719 4.192a.751.751 0 0 1-1.088.791L8 12.347l-3.766 1.98a.75.75 0 0 1-1.088-.79l.72-4.194L.818 6.374a.75.75 0 0 1 .416-1.28l4.21-.611L7.327.668A.75.75 0 0 1 8 .25Zm0 2.445L6.615 5.5a.75.75 0 0 1-.564.41l-3.097.45 2.24 2.184a.75.75 0 0 1 .216.664l-.528 3.084 2.769-1.456a.75.75 0 0 1 .698 0l2.77 1.456-.53-3.084a.75.75 0 0 1 .216-.664l2.24-2.183-3.096-.45a.75.75 0 0 1-.564-.41L8 2.694Z\"></path>\n</svg>\n 627 stars today\n</span> </div>",
"stars_today": 627
},
"repo_detail_raw": {
"id": 915520679,
"url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch",
"fork": false,
"name": "train-llm-from-scratch",
"size": 85,
"forks": 437,
"owner": {
"id": 63067900,
"url": "https://api.github.com/users/FareedKhan-dev",
"type": "User",
"login": "FareedKhan-dev",
"node_id": "MDQ6VXNlcjYzMDY3OTAw",
"html_url": "https://github.com/FareedKhan-dev",
"gists_url": "https://api.github.com/users/FareedKhan-dev/gists{/gist_id}",
"repos_url": "https://api.github.com/users/FareedKhan-dev/repos",
"avatar_url": "https://avatars.githubusercontent.com/u/63067900?v=4",
"events_url": "https://api.github.com/users/FareedKhan-dev/events{/privacy}",
"site_admin": false,
"gravatar_id": "",
"starred_url": "https://api.github.com/users/FareedKhan-dev/starred{/owner}{/repo}",
"followers_url": "https://api.github.com/users/FareedKhan-dev/followers",
"following_url": "https://api.github.com/users/FareedKhan-dev/following{/other_user}",
"user_view_type": "public",
"organizations_url": "https://api.github.com/users/FareedKhan-dev/orgs",
"subscriptions_url": "https://api.github.com/users/FareedKhan-dev/subscriptions",
"received_events_url": "https://api.github.com/users/FareedKhan-dev/received_events"
},
"topics": [
"gemini",
"large-language-models",
"llm",
"openai",
"training",
"transformers"
],
"git_url": "git://github.com/FareedKhan-dev/train-llm-from-scratch.git",
"license": {
"key": "mit",
"url": "https://api.github.com/licenses/mit",
"name": "MIT License",
"node_id": "MDc6TGljZW5zZTEz",
"spdx_id": "MIT"
},
"node_id": "R_kgDONpG8pw",
"private": false,
"ssh_url": "git@github.com:FareedKhan-dev/train-llm-from-scratch.git",
"svn_url": "https://github.com/FareedKhan-dev/train-llm-from-scratch",
"archived": false,
"disabled": false,
"has_wiki": true,
"homepage": "https://medium.com/@fareedkhandev/building-a-2-billion-parameter-llm-from-scratch-using-python-1325cb05d6fb",
"html_url": "https://github.com/FareedKhan-dev/train-llm-from-scratch",
"keys_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/keys{/key_id}",
"language": "Jupyter Notebook",
"tags_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/tags",
"watchers": 2892,
"blobs_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/git/blobs{/sha}",
"clone_url": "https://github.com/FareedKhan-dev/train-llm-from-scratch.git",
"forks_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/forks",
"full_name": "FareedKhan-dev/train-llm-from-scratch",
"has_pages": false,
"hooks_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/hooks",
"pulls_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/pulls{/number}",
"pushed_at": "2026-05-22T04:17:43Z",
"teams_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/teams",
"trees_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/git/trees{/sha}",
"created_at": "2025-01-12T04:03:05Z",
"events_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/events",
"has_issues": true,
"issues_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/issues{/number}",
"labels_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/labels{/name}",
"merges_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/merges",
"mirror_url": null,
"updated_at": "2026-05-31T21:58:39Z",
"visibility": "public",
"archive_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/{archive_format}{/ref}",
"commits_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/commits{/sha}",
"compare_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/compare/{base}...{head}",
"description": "A straightforward method for training your LLM, from downloading data to generating text.",
"forks_count": 437,
"is_template": false,
"open_issues": 5,
"branches_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/branches{/branch}",
"comments_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/comments{/number}",
"contents_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/contents/{+path}",
"git_refs_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/git/refs{/sha}",
"git_tags_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/git/tags{/sha}",
"has_projects": true,
"releases_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/releases{/id}",
"statuses_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/statuses/{sha}",
"allow_forking": true,
"assignees_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/assignees{/user}",
"downloads_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/downloads",
"has_downloads": true,
"languages_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/languages",
"network_count": 437,
"default_branch": "main",
"milestones_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/milestones{/number}",
"stargazers_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/stargazers",
"watchers_count": 2892,
"deployments_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/deployments",
"git_commits_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/git/commits{/sha}",
"has_discussions": false,
"subscribers_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/subscribers",
"contributors_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/contributors",
"issue_events_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/issues/events{/number}",
"stargazers_count": 2892,
"subscription_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/subscription",
"temp_clone_token": null,
"collaborators_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/collaborators{/collaborator}",
"has_pull_requests": true,
"issue_comment_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/issues/comments{/number}",
"notifications_url": "https://api.github.com/repos/FareedKhan-dev/train-llm-from-scratch/notifications{?since,all,participating}",
"open_issues_count": 5,
"subscribers_count": 34,
"web_commit_signoff_required": false,
"pull_request_creation_policy": "all"
},
"readme_raw": {
"summary": "<div align=\"center\" Train LLM From Scratch I am Looking for a PhD position in AI . GitHub",
"raw_text": "\n\n<div align=\"center\">\n\n<!-- omit in toc -->\n# Train LLM From Scratch\n \n   [](#step-by-step-code-explanation)\n\n**I am Looking for a PhD position in AI**. [GitHub](https://github.com/FareedKhan-dev)\n\n</div>\n\nI implemented a transformer model from scratch using PyTorch, based on the paper [Attention is All You Need](https://arxiv.org/abs/1706.03762). You can use my scripts to train your own **billion** or **million** parameter LLM using a single GPU.\n\nBelow is the output of the trained 13 million parameter LLM:\n\n```\nIn ***1978, The park was returned to the factory-plate that \nthe public share to the lower of the electronic fence that \nfollow from the Station's cities. The Canal of ancient Western \nnations were confined to the city spot. The villages were directly \nlinked to cities in China that revolt that the US budget and in\nOdambinais is uncertain and fortune established in rural areas.\n```\n<!-- omit in toc -->\n## Table of Contents\n- [Training Data Info](#training-data-info)\n- [Prerequisites and Training Time](#prerequisites-and-training-time)\n- [Code Structure](#code-structure)\n- [Usage](#usage)\n- [Step by Step Code Explanation](#step-by-step-code-explanation)\n - [Importing Libraries](#importing-libraries)\n - [Preparing the Training Data](#preparing-the-training-data)\n - [Transformer Overview](#transformer-overview)\n - [Multi Layer Perceptron (MLP)](#multi-layer-perceptron-mlp)\n - [Single Head Attention](#single-head-attention)\n - [Multi Head Attention](#multi-head-attention)\n - [Transformer Block](#transformer-block)\n - [The Final Model](#the-final-model)\n - [Batch Processing](#batch-processing)\n - [Training Parameters](#trai",
"raw_text_truncated": true
},
"contributors_raw": {
"truncated": false,
"contributors": [
{
"type": "User",
"login": "FareedKhan-dev",
"html_url": "https://github.com/FareedKhan-dev",
"contributions": 22
},
{
"type": "User",
"login": "TianyiQ",
"html_url": "https://github.com/TianyiQ",
"contributions": 19
},
{
"type": "User",
"login": "eltociear",
"html_url": "https://github.com/eltociear",
"contributions": 1
},
{
"type": "User",
"login": "Jah-yee",
"html_url": "https://github.com/Jah-yee",
"contributions": 1
},
{
"type": "User",
"login": "its-not-rocket-science",
"html_url": "https://github.com/its-not-rocket-science",
"contributions": 1
},
{
"type": "User",
"login": "konglingfan",
"html_url": "https://github.com/konglingfan",
"contributions": 1
}
]
},
"funding_raw": {
"path": null,
"exists": false,
"content": null
},
"stats_raw": {
"forks_total": 437,
"stars_today": 627,
"stars_total": 2892,
"watchers_count": 2892,
"open_issues_count": 5
},
"aux_raw": {
"selected_fields": {
"topics": [
"gemini",
"large-language-models",
"llm",
"openai",
"training",
"transformers"
],
"is_fork": false,
"license": "MIT",
"language": "Jupyter Notebook",
"owner_type": "User",
"forks_total": 437,
"has_funding": false,
"is_archived": false,
"owner_login": "FareedKhan-dev",
"stars_today": 627,
"stars_total": 2892,
"homepage_url": "https://medium.com/@fareedkhandev/building-a-2-billion-parameter-llm-from-scratch-using-python-1325cb05d6fb",
"default_branch": "main",
"last_pushed_at": "2026-05-22T04:17:43Z",
"readme_summary": "<div align=\"center\" Train LLM From Scratch I am Looking for a PhD position in AI . GitHub",
"repo_full_name": "FareedKhan-dev/train-llm-from-scratch",
"watchers_count": 2892,
"last_updated_at": "2026-05-31T21:58:39Z",
"top_contributors": [
{
"login": "FareedKhan-dev",
"contributions": 22
},
{
"login": "TianyiQ",
"contributions": 19
},
{
"login": "eltociear",
"contributions": 1
},
{
"login": "Jah-yee",
"contributions": 1
},
{
"login": "its-not-rocket-science",
"contributions": 1
}
],
"contributor_count": 6,
"funding_platforms": [],
"open_issues_count": 5,
"days_since_created": 504,
"created_at_on_source": "2025-01-12T04:03:05Z",
"days_since_last_push": 9,
"top_contributor_share": 0.489
}
},
"selection_meta": {
"readme_status": "ok",
"funding_status": "ok",
"missing_enrichment": [],
"repo_detail_status": "ok",
"contributors_status": "ok"
},
"created_at": "2026-05-31T22:00:33.285Z",
"updated_at": "2026-05-31T22:00:33.285Z"
}