Basic Linux File system

Linux Filesystem Structure (FHS – Filesystem Hierarchy Standard)

The Linux filesystem is a tree structure starting from the root / directory. All files and directories branch from here.

Let’s walk through the important directories, what they’re for, and give real examples relevant to data science and system usage.

---

📁 / – Root Directory

• Top-level directory. Everything starts here.

• Owned by root user (/root is not the same as /).

---

📁 /bin – Essential User Binaries

• Contains essential executable commands needed for basic use.

• Used during boot and single-user mode.

🔧 Examples:

bash

CopyEdit

/bin/ls       # list files
/bin/cp       # copy
/bin/mkdir    # make directory
/bin/bash     # shell

---

📁 /sbin – System Binaries

• Contains admin commands for system maintenance.

• Requires sudo or root privileges.

🔧 Examples:

bash

CopyEdit

/sbin/ifconfig    # view network interfaces (older)
/sbin/reboot      # restart system
/sbin/mkfs        # create filesystem

---

📁 /usr – User Programs & Data

Think of /usr like "user space" apps. It’s usually the largest directory.

• /usr/bin – non-essential command binaries (installed packages)

• /usr/sbin – admin binaries not needed for basic system

• /usr/lib – libraries for installed programs

• /usr/local – user-installed software

🔧 Examples:

bash

CopyEdit

/usr/bin/python3
/usr/bin/Rscript
/usr/lib/python3/dist-packages/   # Python libraries
/usr/local/bin/jupyter            # Jupyter Notebook if installed manually

📦 Installations from apt/pip often land here.

---

📁 /etc – Configuration Files

• All system-wide configuration files.

• Plain text, editable with editors like nano, vim.

🔧 Examples:

bash

CopyEdit

/etc/passwd          # user account info
/etc/hostname        # system name
/etc/network/interfaces   # network config
/etc/mysql/my.cnf    # MySQL config
/etc/jupyter/jupyter_notebook_config.py

🧠 Good place to explore how your system is configured.

---

📁 /home – User Home Directories

• Each user has a folder: /home/sree, /home/ubuntu, etc.

• This is where you do your work as a regular user.

🔧 Examples:

bash

CopyEdit

/home/sree/projects/data-cleaning/
~/notebooks/eda.ipynb

💡 Usually where:

• Git repos are cloned

• Virtual environments live

• Jupyter notebooks are saved

---

📁 /var – Variable Data

• Files that change frequently: logs, caches, spool files.

🔧 Examples:

bash

CopyEdit

/var/log/syslog             # system logs
/var/log/jupyter.log        # Jupyter logs (if configured)
/var/cache/apt/archives     # downloaded .deb packages
/var/lib/mysql              # MySQL databases

🧠 Check logs here for troubleshooting issues.

---

📁 /tmp – Temporary Files

• Temporary storage for files.

• Automatically cleared on reboot.

🔧 Examples:

bash

CopyEdit

/tmp/tmp1234.csv

💡 Can be used for storing large intermediate files in pipelines.

---

📁 /dev – Device Files

• Interface to hardware (not actual files).

• Devices like USB, disks, GPU appear here.

🔧 Examples:

bash

CopyEdit

/dev/sda     # primary hard disk
/dev/nvidia0 # GPU device for CUDA

💡 Useful when working with large data storage devices or GPUs.

---

📁 /proc – Kernel & Process Info (Virtual FS)

• Not real files — gives a view into the kernel.

• Contains a folder for each running process, by PID.

🔧 Examples:

bash

CopyEdit

/proc/cpuinfo         # CPU details
/proc/meminfo         # Memory usage
/proc/1234/status     # Info about PID 1234

---

📁 /lib, /lib64 – Shared Libraries

• Contains dynamic libraries (.so files) used by binaries in /bin and /sbin.

🔧 Examples:

bash

CopyEdit

/lib/x86_64-linux-gnu/libc.so.6

💡 Think of .so files like .dll files in Windows.

---

📁 /opt – Optional Software

• For third-party applications not from package manager.

🔧 Examples:

bash

CopyEdit

/opt/anaconda/

💡 Often used to install Anaconda, VS Code Server, MATLAB, etc.

---

📁 /root – Root User's Home

• Home directory for the root user (different from /).

• Has full privileges; used only for admin work.

🔧 Examples:

bash

CopyEdit

/root/.bashrc
/root/admin-scripts/

When You Install python3-pip, Where Does It Get Installed?

🧰 Step-by-Step:

When you install python3-pip via APT (the default package manager on Debian/Ubuntu):

bash

CopyEdit

sudo apt update
sudo apt install python3-pip

✅ Here's what happens:

1. Binary (executable):

• Installed at:

bash

CopyEdit

/usr/bin/pip3

This is the actual pip3 command you run in terminal. You can verify:

bash

CopyEdit

which pip3
# Output:
/usr/bin/pip3

---

2. Python Modules Installed via pip3:

If you then install a Python package using:

bash

CopyEdit

pip3 install numpy

Depending on the environment, the packages go into:

• System-wide:

bash

CopyEdit

/usr/lib/python3/dist-packages/

or

bash

CopyEdit

/usr/local/lib/python3.X/dist-packages/

Check it with:

bash

CopyEdit

pip3 show numpy

---

⚠️ Important Notes:

Situation | Where packages go -- | -- APT-installed pip | /usr/bin/pip3, packages in /usr/lib/ or /usr/local/lib/ pip installed via get-pip.py or python3 -m ensurepip | May go to /usr/local/bin/pip3 Inside a virtual environment | Everything is isolated under venv/ directory As a user (--user flag) | Installed in your home directory: ~/.local/bin/ and ~/.local/lib/

---

🧪 Try These Yourself:

bash

CopyEdit

which pip3
pip3 --version
pip3 list
pip3 show pandas

This gives you both the location and how it’s connected to your Python interpreter.