Skip to content

Welcome to Second Brain 2022 Projects 4 Vs Of Big Data About Static Abstract Class And Interface Activation Function Activation Functions Adam Advent Of Code Adversarial Attack Ag Grid Agile Travel Game AI Agents And World Models Ai First Software Delivery Ai Ml Roadmap Ai Resources Allotted Time For Tasks Based On Tags On Habitica Android App Ideation Android App Tasks Android Code Snippet Android Studio Build System Android Toast Example Android Angular Apache Kafka Apache Application Architecture Apps To Download Artificial Neural Network Associating Specific Music To Specific Tasks Atomic Designing Methodology Attendance Tracking System Attention Is All You Need Authentication Backend Dev Bag Of Words Basics Of Os Bcgx Interview Best Time To Buy And Sell Stocks Big Data Analytics Blue Green Deployment Bootstrapping Browser Bookmarks Buddy Feedback Bug Bash Build Files Notes Building A Portfolio Website Building A Second Brain C Basics Cache Types Capsule Network Cash Register Challenge With Initial Approach Change Password Chess Readme Chess Tournament Msg Ci Cd Ct Ci Cd Class Components Classes Schedule Claude Setup Client Onboarding Call Climbing Stairs Cloud Computing Cloudwatch Overview Cnn Code Smells Code Walkthrough Tl Coding Interview Preparation Cold Calls Hr Colima Competitive Programming Guide Complete Web Development Bootcamp Computer Networks Configuring Webstorm For React Connect For Code Setup Connext Quick Points Consciousness Universe Questions Contacts Containerisation Contains Duplicate Content Delivery Network Contracts Convolution Convolutional Neural Network Cookieslocal Storagesession Storage Cornell Notes System Corpus Counter App Course Schedule Creating A Virtual Dom Cross Functional Requirements Crypto Startup Prep Focus Crypto Startup R1 Resources Crypto Startup Css Grid Css Battles Css Games Cuda Cores Cyber Attacks Dao Vs Dto Data Analysis Data Processing Data Visualization With Seaborn Data Bricks Data Drift Data Engineering Session 1 Data Engineering Session Data Lake Data Lakehouse Data Science Notes Database Management Systems Database Migration Types Database Paradigms Database Day 2 1 Day 4 1 Day 7 1 Dealing With Feedback Deep Agile Deepstream Densenet Design And Engineering Of Computer Systems Design App Prompt Developer Practice Features Development Tools Device Driver Difference Between Mac And Windows Docker Storage Docker Commands Docker File Docker Domain Driven Design Domain Layer Driverless Ai Dynamic Array Dynamic Programming Edge Computing Edge Ml Edge Tpu Eks Cluster Embedded Machine Learning Coursera Encoder Decoder Ethical Impacts Of Ai Example Prompt Exercism Exercises Exploding Gradient Extreme Programming Xp And Test Driven Development Tdd Factorial Steps Failure Is The Stepping Stone To Success Faiss Fang2018 Fast Api Federated Learning Fibonacci Sequence Final Code Find All Numbers Disappeared In Array Finding Problems While Learning To Code First Steps In Java Udemy Course Fixed Length Flow + Career Fnv Tracability App Composables Followup Post Ideas For Building A Second Brain Forge Week4 Formik From Cpu To Browser Frontend Dev Notes Frontend Heavy Questions Frontend Interview Prep Frontend Sde Prep Frontend Startups Companies Frontend System Design Functional Api Functional Programming Functional Reactive Programming Game Programming Patterns Gang Of Four Design Patterns General Structure Generative Adversarial Network Google Cloud Data Engineer Google Automl Google Coral Google Disco Build Apps With Prompts Gpt 3 Gradient Descent Graph Neural Network Grondwolf Client Meeting Groundwolf Application Groundwolf Final Feedback Gru Gurushifu H5 Format Hack The Box Challenges Hackathon Ideas Hadoop Hardware Accelerator Histograms Hooks How To Be A 100x Engineer Using Ai Notes How And Why To Integrate Often How Does Linux Boot Process Work My Notes How Does Linux Boot Process Work How Google Script Is Converted To Javascript How I Use Llm How To Become A Cult Leader Image Preprocessing Imperative Vs Declarative Build Tools Initial Learnings Typescript Initial Code Initial Learnings Typescript Inside The Python Virtual Machine Integration Test Intro To Operating System Intro To Oss Introduction To Bits And Bytes Introduction To Computer Networks Introduction To Docker Compose Introduction To Operating System Introduction To Tensorflow Udacity Invest Session Ios Icon Customiser Iteration 2 Planning Iteration 3 Ipm Java 1 Java 8 Tech Talk Java Notes Java Optional Class Java Stream Javascript Es6 Playlist Javascript Executor Selenium Javascript Flashcards Javascript Optional Chaining Javascript Jax Json Format Keras Kernel Definition Key Value Pairs Kubernetes Labeling Py Langchain.Js Language Translation Learn Anything Gpt Prompt Learn Css Codecademy Learn With Chatgpt Learning Checklist Learning Log Lecture 10 Virtual Machines And Containers Leetcode Getting Started Leetcode Guide Leetcode Patterns Leetcode Strategy Linux Llm Parser Long Form Lstm Mac Installations Machine Learning Engineer JD Machine Learning Mars Rover Maximum Subarray Meditation Book Review Microservices Migrations Missing Number Ml In Production Coursera Ml Kit ML Notes Ml Ops Roadmap Ml Pipeline Ml Production Pipelines Mlflow Mlops Model Conversion Model Deployment Edge Model Drift Module Loading Mongodb Multilingual Language Model My Simple Workflow While Learning Something New Mysql Navigating Gcp Neetcode Dsa Prep Nestjs Overview Net New Jobs Next.Js Nextauth Ngrok Nlp Nodejs Alias Module Notes Np Argmax Npm Libraries Numberai Nvidia Ai Playground Object Oriented Programming (OOP)Object Oriented Programming Concepts Object Oriented Programming On Optimizing React Performance On Ai Agents One Shot Learning Onnx Open Souce Ai Models Opencv Operating System Optimizer Orm Os Concepts Paper Parking Lot Dojo Path To Production Pathways Photography Resources Phpsmallestoftwonumbers Physics Projects Pii Data Pillow Pooling Portfolios Post Training Quantization Processes Productivity Program Learnings Continue When Bored Program Pre Work Tasks Progression Systems Project Chess Verse Project Paper Content Projects To Pursue Promiseall Protobuf Pub Sub Messaging Model Pytorch Quantization Aware Training Quantization Questions Random Springboot Notes React Interview 1 React Interview 2 React Interview React Code React Controlled Component React Framework React Notes 1 React Notes 2 React Notes 3 React Notes 4 React Notes To Refresh React Tdd Session React Test Driven Development React.Gg Visualized Notes React Rectangle Dojo Rectangle Recurrent Neural Network Redux Toolkit Redux Relevant Techs Relu Remix Remote Procedure Callsrpc Requirements > Architecture Research Papers Resources 1 Resources Rest Api Definition Rest Constriants Restful Web Services Spring Framework Resume Rnn Round 1 Rtk Query S3 Bucket Overview Scratch Project Scripting Vs Programming Sequential Api Sequential Shallow Vs Render Vs Mount Side Gigs Sigmoid Single Number Slow Down To Go Faster Slugs Soap Protocol Softmax Software Design Principles Solid Principles 1 Solid Principles Spring Loose Coupling Springboot Annotations Springboot Mvc Springboot Notes Springboot Tests Sql Vs Mysql Startup R0 Startup Streamlit Supervised Learning Swr Stale While Revalidation System Design Questions Tanh Tanstack Query Task Tracker Api Tech Task Notes Tdd Feedback Tdd Tic Tac Toe Technology Stack Technology Tensor Cores Tensorflow Extended Tensorflow Lite Tensorflow Resources Tensorflow Tensorrt Testing Certifications Testing Text To Handwriting Tflite Model Maker Tfx The Eu Ai Act The Singularity Is Nearer My Notes The Valluvar Speaks The Why Thought Task Term Tinyml Book To Learn In Scrimba Tpu Trainee Guidelines Transfer Learning Transformer Transformers Translation Challenge Code Transpiler Tribonacci Series Two Sum Js Types Of Data Typescript Ui Discussion Team Lead Uipath Unknown User Story Netflix Using Webassemply In React Vanilla Javascript Vanishing Gradient Variational Autoencoder Wallet Design Patterns Wallet Implementation Principles Web Resources Websites 1 Weights And Biases What Is Docker Work Summary Working Of A LLM Write A System Call Notes Xss References Zustand

Vanishing Gradient

vanishing-gradient

Gradient values become too small during backpropagation → network stops learning

Common in deep rnn → weights shrink exponentially

Solutions:

lstm gates
gru architecture
ReLU activation
Residual connections

*References

exploding-gradient
Vanishing Gradient Problem
Why LSTM Works

*References

#ml-notes

Vanilla Javascript

Previous Page

Variational Autoencoder

Next Page

On this page

vanishing-gradient *References *References