Data Handlingالتعامل مع البيانات

L21 — Setting Up Python, Jupyter & NumPy: Arrays, Shapes & dtypes

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U06 Batch L21–L23 v1 · Content date: 15 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13 and NumPy 2.5) on Linux and on Windows 11 on 4 November 2026; on Windows 11 the environment was created with the install line in Part A (see the Code Test Report). macOS and Python 3.14 were not install-tested. If a step does not work on your computer, use the Support fallback in Section 4.

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	Setting Up Python, Jupyter & NumPy: Arrays, Shapes & dtypes
Stage	Stage 2 — Data Handling
Unit	U06 — NumPy Foundations
Lesson ID	L21 (first lesson of U06 and of Stage 2)
Skills	Arrays; shapes
Difficulty	Beginner in NumPy (Stage 1 Python assumed)
Estimated time	About 3 hours (planning estimate, unvalidated). This includes about 45–60 minutes of one-time setup. Setup time varies widely by device and operating system.
Assessment level	Understanding → Practice
Prerequisites	Stage 1. From U04: variables and int/float/bool/str (L13), comparisons (L14), lists, loops and zero-based indexing (L15), functions, error messages and assert (L16). From U05: expected answer first, tracing and testing (L19). From U02: revenue, cost, gross profit and margin (L06).
New in this lesson	import, packages, virtual environments, notebooks, tuples, files and folders on the command line. You do not need to know these already.

Time breakdown (planning estimate):

• Setup (Section 4, Part A): 45–60 min.
• Notebook basics: 15 min.
• Concept and worked example: 35 min.
• Guided practice: 25 min.
• Independent practice: 15 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 10 min (optional).
• Workplace task and mini challenge: 20 min.

What you need:

• A computer where you can install software, or a hosted Jupyter environment (see "If setup does not work").
• Internet access once, to download Python and the packages.
• No account, no API key and no paid software. Nothing in Stage 2 requires them.

---

2. Learning Objective

By the end of this lesson, you will be able to set up a working Stage 2 environment, show its versions in a notebook, and create 1-D and 2-D NumPy arrays whose shape, ndim, size and dtype you predict correctly before running the code.

D.World approved objective (verbatim): Set up a working virtual environment with Python 3.13-3.14, NumPy, pandas, openpyxl and JupyterLab, show the printed versions in a notebook, and create 1-D and 2-D NumPy arrays whose shape, ndim, size and dtype are predicted correctly before running the code.

---

3. Why It Matters

In U04 you stored a week of sales in a Python list and used a loop to add it up. That works for one list. Real work rarely stays that small. Cedar Lane, a fictional stationery shop, now has three pickup points. Each one keeps its own list of daily unit sales. To compare them, someone copies numbers between lists by hand, and mistakes creep in.

NumPy stores all of those numbers in one grid, called an array. Each row is a pickup point and each column is a day. Once the data is in a grid, a single line of code can do a calculation on every value at once. Every later Stage 2 tool is built on this idea. In U07, each column of a pandas table is a labelled NumPy-style array.

Career context. The D.World Curriculum Map lists NumPy as part of the software that current Data Scientist work asks for. This is background on why the skill is in the curriculum. It is not a promise of any job or outcome.

Before any of that, you need a reliable place to run code. This lesson sets up the environment that all of Stage 2 uses, so later lessons install nothing new.

Key principle: predict the shape before you run the code. If you cannot say what the grid should look like, you cannot tell whether the computer built it correctly.

---

4. Concept

Part A — Set up the Stage 2 environment (do this once)

The pieces and why each one is needed

Piece	What it is	Why Stage 2 needs it
Python 3.14 (3.13 accepted)	The language you used in U04	NumPy 2.5 supports Python 3.12–3.14. D.World uses 3.14 for new setups and accepts 3.13.
Package	Extra code someone else wrote and published, installed with pip	NumPy, pandas and openpyxl are packages. They are not part of Python itself.
Virtual environment ("venv")	A separate folder with its own Python and its own packages	It keeps the Stage 2 versions together and stops them clashing with other projects.
NumPy 2.5.x	Arrays and fast numerical work	U06
pandas 3.0.x	Tables (DataFrames)	U07 onwards. Installed now so that later lessons install nothing new.
openpyxl 3.1.5	Lets pandas read and write Excel .xlsx files	U07 and U08
JupyterLab 4.6.x	A browser-based editor for notebooks	Stage 2 work is submitted as notebooks. The classic Notebook 7.6.x is also acceptable.

Do not use Python 3.15 for Stage 2 yet, even though it was released on 9 October 2026. On 15 October 2026, NumPy 2.5's release notes listed support only up to Python 3.14. Do not use 3.12 or older for a new setup either. If you installed an older Python in L13, keep it, and add 3.14 alongside it.

"x" in a version means "any number here". For example, 2.5.x means 2.5.0, 2.5.1, 2.5.4 and so on. The command below asks pip for the newest release in that series.

Step 1 — Install Python 3.14

Open a terminal: Terminal on macOS and Linux, PowerShell or Command Prompt on Windows. A terminal is a window where you type commands and press Enter.

Windows. Python's Windows documentation says: "To obtain Python from the CPython team, use the Python Install Manager." Download it from python.org/downloads or from the Microsoft Store. Then, in PowerShell:

py install 3.14
py -V:3.14 --version

• The second command should print Python 3.14. followed by a number.
• If the installer offers to add a folder to your PATH, you may accept. It is optional, because the py command works without it.
• Python's documentation notes that the older full installer is deprecated as of 3.14. Use the install manager unless D.World Support tells you otherwise.

macOS. Download the macOS installer package (.pkg) for Python 3.14 from python.org/downloads/macos. Then:

1. Double-click the package and follow the installer. An administrator account is needed.
2. Afterwards, open the /Applications/Python 3.14/ folder. Double-click Install Certificates.command. Without this step, downloads from Python can fail with certificate errors.
3. In Terminal, check:

python3.14 --version

Do not change or delete Apple's own /usr/bin/python3.

Linux. First check what you already have:

python3 --version
python3.14 --version

• If either command shows 3.14 or 3.13, use that command in Step 2.
• If not, install Python 3.14 or 3.13 with your distribution's package manager, if it offers one of them. Otherwise use the fallback in "If setup does not work" below.
• On Debian and Ubuntu, the venv module can be a separate package. If Step 2 says ensurepip is not available, install the package named in that message (for example python3-venv), then repeat Step 2.

Step 2 — Create a project folder and a virtual environment

Make a folder for Stage 2, for example dworld-stage2, and move into it. Then create an environment called dworld-env:

System	Commands
Windows (PowerShell)	mkdir dworld-stage2 · cd dworld-stage2 · py -V:3.14 -m venv dworld-env
macOS	mkdir dworld-stage2 · cd dworld-stage2 · python3.14 -m venv dworld-env
Linux	mkdir dworld-stage2 · cd dworld-stage2 · python3.14 -m venv dworld-env (or python3.13 ...)

Nothing is printed when it works. A new folder dworld-env appears.

Step 3 — Activate the environment

Activating means "use this environment's Python and pip in this terminal window". You must do it every time you open a new terminal for Stage 2.

System / shell	Command
macOS or Linux (bash/zsh)	source dworld-env/bin/activate
Windows Command Prompt	dworld-env\Scripts\activate.bat
Windows PowerShell	dworld-env\Scripts\Activate.ps1

When it works, the prompt usually starts with (dworld-env).

PowerShell blocked the script? Windows may refuse to run Activate.ps1. The Python documentation describes a setting called the "execution policy" that controls this. Do not change security settings on a shared or work computer without permission. The simplest fix is to use Command Prompt and activate.bat instead. You can also ask D.World Support.

Step 4 — Install the packages

With the environment active, type this as one line:

python -m pip install "numpy==2.5.*" "pandas==3.0.*" "openpyxl==3.1.5" "jupyterlab==4.6.*"

• The same line works on Windows, macOS and Linux once the environment is active.
• Use python -m pip rather than just pip. It makes sure the packages go into the Python you are actually using.
• Alternative: the practice files include requirements-stage2.txt, which contains the same four lines. Install from it with python -m pip install -r requirements-stage2.txt.
• Installation downloads files and can take several minutes.

Step 5 — Start JupyterLab and create a notebook

With the environment still active:

jupyter lab

1. Your browser opens JupyterLab. Keep the terminal window open while you work; closing it stops JupyterLab.
2. In JupyterLab, click the + button in the file browser. A Launcher tab opens.
3. Under Notebook, choose the Python 3 kernel (it is usually labelled Python 3 (ipykernel)). Because you started JupyterLab from the active environment, this kernel uses dworld-env.
4. Rename the notebook L21_setup_and_arrays.ipynb: right-click its tab or its name in the file browser.
5. To stop JupyterLab later, save your work, then press Ctrl + C in the terminal and confirm.

Next time: open a terminal → go to dworld-stage2 → activate → jupyter lab. You do not need to install again.

If setup does not work — the Support fallback

Setup problems are common, and they are not a sign that you cannot do data science. If you are stuck for more than about 30 minutes:

1. Copy the last line of the error message.
2. Remove your user name and folder names from it (see Section 10).
3. Send it to D.World Support.

D.World Support (support.dworld@gmail.com or WhatsApp, listed on the Support page) will help you fix the setup from that line. If your own computer cannot run the setup, you can use any hosted Jupyter service you have access to, run the install line from Part A in its first cell, and check the versions with the version cell. Using it is fully acceptable for all of Stage 2. If you cannot use Jupyter at all, D.World Support records an exception. A Python script is then accepted for U06–U08 work. The Project 2 notebook will still be produced in Jupyter (your own or a hosted one).

Part B — Notebook basics

A notebook is a document made of cells:

Cell type	Contains	Run it with
Code cell	Python code. Its output appears directly below it.	Shift + Enter (runs the cell and moves to the next)
Markdown cell	Notes and headings in plain text	Shift + Enter shows it formatted

Behind each notebook runs a kernel: the Python process that executes your code and remembers every variable you have created. That memory causes the most common notebook mistake.

Hidden state. Suppose you run a cell that sets price = 12, then delete that cell. The kernel still remembers price, so later cells keep working. When someone else opens your notebook, price does not exist and their run fails. Running cells out of order causes the same problem. The number in [ ] next to each cell shows the order in which cells actually ran.

The fix, and a Stage 2 rule. Before you submit or share any notebook, use Kernel → Restart Kernel and Run All Cells…. This clears the kernel's memory and runs every cell from top to bottom. If it finishes without an error, the notebook reproduces from scratch. Every Stage 2 notebook submission must pass this check.

In a notebook, if the last line of a cell is an expression, Jupyter displays its value even without print(). Arrays then appear as array([12, 15, ...]). This lesson uses print() so the output looks the same in a notebook and in a script.

Part C — Importing NumPy

In U04 everything you used was built into Python. A package must be imported before use:

import numpy as np

• import numpy loads the package.
• as np gives it a short alias. Writing np instead of numpy is a near-universal convention; you will see it in documentation and in the code of other people.
• After this line, NumPy tools are written as np.something, for example np.array.
• Run the import cell first. If you see NameError: name 'np' is not defined, the import cell has not run in this kernel yet (hidden state again).

Part D — Tuples, briefly

A tuple is like a list written with round brackets: (3, 7). You will not build tuples yourself in U06. You need to read them, because NumPy reports an array's shape as a tuple.

Written	Meaning
(3, 7)	Two numbers: 3 rows, 7 columns
(7,)	One number: 7. The trailing comma is what makes it a tuple of length 1.
()	No dimensions. You will see it for a single value.

Unlike a list, a tuple cannot be changed after it is created.

Part E — From list to array

import numpy as np

north_list = [12, 15, 11, 14, 18, 25, 22]   # Mon..Sun units, North point (synthetic)
north = np.array(north_list)

print(north_list)
print(north)
print(type(north))

[12, 15, 11, 14, 18, 25, 22]
[12 15 11 14 18 25 22]
<class 'numpy.ndarray'>

When you print an array, it shows no commas. Its type is numpy.ndarray, short for "N-dimensional array".

List vs array: the two differences that matter now

	Python list	NumPy array
Values	Can mix types: [12, "Mon", True]	One dtype for all elements
* 2	Repeats the list	Multiplies every element (element-wise)

print(north_list * 2)
print(north * 2)

[12, 15, 11, 14, 18, 25, 22, 12, 15, 11, 14, 18, 25, 22]
[24 30 22 28 36 50 44]

The same symbol means two very different things, so always know which kind of object you have.

Part F — Shape, ndim, size and dtype

Every array has four attributes you will check constantly. Note that there are no brackets after them: they are attributes, not function calls.

Attribute	Question it answers	north
.shape	How many elements along each dimension? (a tuple)	(7,)
.ndim	How many dimensions?	1
.size	How many elements in total?	7
.dtype	What type is every element?	int64

print(north.shape, north.ndim, north.size, north.dtype)

(7,) 1 7 int64

A 2-D array: rows and columns with business meaning. A list of lists, where each inner list has the same length, becomes a grid. Here, rows = pickup points (North, Central, Harbor) and columns = days (Mon … Sun):

units = np.array([
    [12, 15, 11, 14, 18, 25, 22],   # North
    [20, 18, 22, 19, 31, 30, 28],   # Central
    [ 8,  0,  9, 10, 12, 16, 17],   # Harbor
])
print(units)
print(units.shape, units.ndim, units.size, units.dtype)

[[12 15 11 14 18 25 22]
 [20 18 22 19 31 30 28]
 [ 8  0  9 10 12 16 17]]
(3, 7) 2 21 int64

Read the shape as (rows, columns): (3, 7) means 3 rows (pickup points) and 7 columns (days). The order matters. (7, 3) would mean 7 rows and 3 columns, a different layout of the same 21 numbers. size is always the product of the shape: 3 × 7 = 21.

dtypes you will meet in U06

dtype	Holds	Created from
int64	Whole numbers	All-int input
float64	Decimal numbers	Any float in the input, np.zeros, np.ones
bool	True / False	All-bool input, or a comparison (L22)

The 64 means each value uses 64 bits of memory. With NumPy 2.x, the default whole-number dtype is int64 on Windows, macOS and Linux alike.

Upcasting. Because an array has one dtype, NumPy chooses one that can hold every input value. Mixing ints and floats gives float64, and ints are "upcast" to floats. Mixing bool with int gives int64 (True becomes 1, False becomes 0).

mixed = np.array([3, 4.5, 6])
print(mixed, mixed.dtype)
print(np.array([1, True, 0]), np.array([1, True, 0]).dtype)
print(np.array([True, False]).dtype)

[3.  4.5 6. ] float64
[1 1 0] int64
bool

3. means 3.0. NumPy pads short numbers so the columns line up.

Watch out for text. np.array(["12", "15"]) creates an array of text (its dtype starts with <U), not numbers. Arithmetic on it fails. Data that arrives as text must be converted first. You did this for single values in L13; U08 and U09 do it for whole columns.

Part G — Making arrays without typing every value

Function	Makes	Example	Result
np.arange(n)	0, 1, …, n − 1 (like range)	np.arange(7)	[0 1 2 3 4 5 6]
np.arange(a, b)	a, a + 1, …, b − 1	np.arange(1, 8)	[1 2 3 4 5 6 7]
np.zeros(shape)	All zeros, float64	np.zeros((2, 3))	a 2 × 3 grid of 0.
np.ones(shape)	All ones, float64	np.ones(4)	[1. 1. 1. 1.]

For np.zeros and np.ones, a 2-D shape goes inside its own brackets as a tuple: np.zeros((2, 3)). Writing np.zeros(2, 3) is a mistake, because the second number is read as a different setting.

day_numbers = np.arange(1, 8)
placeholder = np.zeros((2, 3))
print(day_numbers, day_numbers.dtype)
print(placeholder)
print(placeholder.shape, placeholder.dtype)

[1 2 3 4 5 6 7] int64
[[0. 0. 0.]
 [0. 0. 0.]]
(2, 3) float64

A placeholder array is useful when you know the layout before you have the data, for example a grid that will be filled in as each month closes.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. Last week, three pickup points sold notebooks. The manager wants all the numbers in one grid, plus a quick check that nothing was lost when the lists were combined.

Step 1 — Expected answer first (before any code).

• 3 pickup points × 7 days → shape (3, 7), ndim 2, size 21.
• All values are whole numbers → dtype int64.
• North's total by hand: 12 + 15 + 11 + 14 + 18 + 25 + 22 = 117.

Step 2 — Build and inspect.

import numpy as np

north_list   = [12, 15, 11, 14, 18, 25, 22]
central_list = [20, 18, 22, 19, 31, 30, 28]
harbor_list  = [ 8,  0,  9, 10, 12, 16, 17]

units = np.array([north_list, central_list, harbor_list])

print("shape:", units.shape)
print("ndim: ", units.ndim)
print("size: ", units.size)
print("dtype:", units.dtype)

shape: (3, 7)
ndim:  2
size:  21
dtype: int64

Step 3 — Compare with the expectation. All four values match. ✔

Step 4 — Cross-check one row with plain Python. units[0] gives the first row, North. Indexing is the topic of L22; here we only use it to check one row. .sum() adds the elements of an array; aggregation is L23.

print(sum(north_list))
print(units[0].sum())
assert units[0].sum() == sum(north_list)
assert units.shape == (3, 7)
print("checks passed")

117
117
checks passed

assert is from L16: it stops with an AssertionError if the condition is false and does nothing when it is true. Putting shape assertions in your notebook means that a wrong layout is caught immediately.

---

6. Visual Explanation

                 columns = days  (axis 1, length 7)
               Mon Tue Wed Thu Fri Sat Sun
             ┌───────────────────────────┐
  row 0 North│ 12  15  11  14  18  25  22│
  row 1 Centr│ 20  18  22  19  31  30  28│   rows = pickup points
  row 2 Harbr│  8   0   9  10  12  16  17│   (axis 0, length 3)
             └───────────────────────────┘
   shape = (3, 7)    ndim = 2    size = 3 × 7 = 21    dtype = int64

   list  [12, 15] * 2  ──►  [12, 15, 12, 15]   (repeats)
   array [12  15] * 2  ──►  [24  30]           (element-wise)

Alt text: A grid with 3 rows and 7 columns. The rows are labelled North, Central and Harbor (pickup points, axis 0, length 3). The columns are labelled Monday to Sunday (days, axis 1, length 7). The values are the weekly unit sales from the worked example. Below the grid: shape (3, 7), ndim 2, size 21, dtype int64. At the bottom, two comparisons. A list [12, 15] times 2 repeats the list, giving [12, 15, 12, 15]. An array [12 15] times 2 multiplies each element, giving [24 30].

---

7. Guided Practice

Work in your notebook L21_setup_and_arrays.ipynb, or open the starter notebook L21_starter.ipynb from the practice files.

Part 1 — Version cell. In the first code cell, type and run:

import sys
import numpy as np
import pandas as pd
import openpyxl
import jupyterlab

print("Python    ", sys.version.split()[0])
print("numpy     ", np.__version__)
print("pandas    ", pd.__version__)
print("openpyxl  ", openpyxl.__version__)
print("jupyterlab", jupyterlab.__version__)

assert sys.version_info[:2] in [(3, 13), (3, 14)], "Use Python 3.13 or 3.14"
assert np.__version__.startswith("2.5."), "Expected numpy 2.5.x"
assert pd.__version__.startswith("3.0."), "Expected pandas 3.0.x"
print("Environment OK")

Your output should look like this. Your numbers after the last dot may differ:

Python     3.14.x
numpy      2.5.x
pandas     3.0.x
openpyxl   3.1.5
jupyterlab 4.6.x
Environment OK

• If an import line fails with ModuleNotFoundError, the notebook is not using dworld-env. Most often, JupyterLab was started without activating the environment first.
• If an assert fails, read its message. It tells you which version is wrong.

Part 2 — From list to array. Turn North's 7-day list into an array.

1. Predict first: write down its shape, ndim, size and dtype in a Markdown cell.
2. Create it, print the four attributes, and compare them with your prediction.
3. Cross-check: compare north.sum() with Python's sum(north_list).

Part 3 — The 3 × 7 grid. Build the units array from the worked example.

1. Predict the four attributes, then print them.
2. In a Markdown cell, write in your own words what one row means and what one column means.

Part 4 — List × 2 vs array × 2. Predict, then run north_list * 2 and north * 2. In one sentence, explain the difference.

Part 5 — Restart and run all. Use Kernel → Restart Kernel and Run All Cells…. Every cell must run from top to bottom without an error. If one fails, fix the order of your cells and repeat. Save the notebook; it is your setup evidence (Section 20).

---

8. Independent Practice

Scenario (synthetic). Cedar Lane wants to track monthly unit sales for 4 products (notebooks, planners, pen sets and sticky notes) over 12 months. The sales have not happened yet, so you will create a placeholder grid of zeros.

1. Without hints, choose the shape. Create the grid with np.zeros, then print its shape, ndim, size and dtype.
2. In a Markdown cell, state what one row means and what one column means in your grid.
3. Explain in one sentence why you chose this order and not the other one. Both are possible; your reason is what counts.
4. Predict the dtype of np.array([120, 45.5, 80, 30]). Write your prediction, then run it to check. Explain why it has that dtype.
5. Predict the size of your placeholder before printing it, and show how you calculated it.

Run Restart Kernel and Run All Cells… before you finish.

No model answer is shown here.

---

9. Knowledge Check — Shape and dtype Prediction

Write each prediction before you run anything. Then run the code to check.

Q1. What does this print?

a = np.array([[5, 1, 0, 2, 7, 3, 4],
              [2, 2, 6, 1, 0, 9, 8],
              [4, 4, 4, 4, 4, 4, 4]])
print(a.shape)

• A) (7, 3)
• B) (3, 7)
• C) 21
• D) (3,)

Q2. A grid has rows = 5 stores and columns = 30 days. Which shape matches?

• A) (30, 5)
• B) (5, 30)
• C) (150,)
• D) (5,)

Q3. What does this print?

weekly = [3, 4]
print(weekly * 2)

• A) [6, 8]
• B) [6 8]
• C) [3, 4, 3, 4]
• D) An error

Q4. What does this print?

weekly = np.array([3, 4])
print(weekly * 2)

• A) [6 8]
• B) [3 4 3 4]
• C) [3, 4, 3, 4]
• D) An error

Q5. What is the dtype of np.array([2, 7, 1.5])?

• A) int64
• B) float64
• C) bool
• D) A mix of int64 and float64

Q6. What does this print?

p = np.zeros((5, 6))
print(p.ndim, p.size, p.dtype)

• A) 2 30 float64
• B) 2 11 int64
• C) 6 30 float64
• D) 2 30 int64

Q7. You deleted a cell that defined price, yet the cells below still run. A classmate opens your notebook and gets NameError: name 'price' is not defined. What is the best explanation and fix?

• A) The classmate has the wrong NumPy version; they should reinstall.
• B) Your kernel still remembered price (hidden state); use Restart Kernel and Run All Cells before sharing, and restore the missing cell.
• C) Notebooks cannot share variables; switch to a script.
• D) price is a reserved word in NumPy.

Q8. What is the shape of np.array([10, 20, 30, 40, 50])?

• A) (5,)
• B) (1, 5)
• C) (5, 1)
• D) 5

---

10. AI in Practice (optional, bounded)

Goal: get a plain-language explanation of a setup error, or of the list-vs-array difference, and then check it yourself.

Option A — An installation or import error.

1. Copy the last line of the error, plus the command you typed.
2. Remove personal details before pasting anything (U03 L12): your user name, folder paths such as C:\Users\amira\... or /Users/amira/..., your computer name, and any e-mail address. Replace them with <USER> or <PATH>.
3. Ask:

"I am a beginner setting up Python 3.14 in a virtual environment on <Windows/macOS/Linux>. I ran this command: <command>. The last line of the error was: <error with personal details removed>. Explain in plain English what this error usually means and give me the two most likely causes. Do not tell me to change security settings or to install anything other than numpy, pandas, openpyxl or jupyterlab."

Option B — List vs array. Ask:

"In Python, why does [3, 4] * 2 give a different result from numpy.array([3, 4]) * 2? Explain for a beginner, with one example of when each behaviour is useful."

Boundaries:

• AI is optional. Every task in this lesson can be done without it.
• Use only synthetic data, your own error messages with personal details removed, or the examples in this lesson.
• Do not paste passwords, keys or real customer data.
• Do not run a command an AI suggests until you understand what it does (Section 11).

---

11. AI Verification

1. Does it match what you saw? The explanation should mention the same error type and the same command. If it describes a different error, it is guessing.
2. Check versions, not opinions. Your version cell is the evidence. After a suggested fix, rerun it. It should print Python 3.13 or 3.14, numpy 2.5.x, pandas 3.0.x, openpyxl 3.1.5 and jupyterlab 4.6.x, and then Environment OK.
3. Reject risky or off-scope advice. Examples: changing security or execution-policy settings on a shared computer; installing with administrator rights when a venv should be used; installing a different Python version (such as 3.15) or extra packages; using another tool than the one this lesson uses. Ask D.World Support instead.
4. Test the list-vs-array claim yourself. Run [3, 4] * 2 and np.array([3, 4]) * 2. The outputs, not the AI's wording, are what you trust.
5. Restart Kernel and Run All Cells after any change. A fix that only works in your current kernel is not a fix.

---

12. Business Lens

One grid makes weekly checks faster. Cedar Lane's three pickup points kept daily sales in separate lists. That means three files to open, three totals to calculate, and three chances to copy a number wrongly.

With one (3, 7) grid:

• The layout is explicit. Anyone reading shape (3, 7) with the note "rows = pickup points, columns = days" knows exactly what each number is.
• A wrong layout is caught early. An assert units.shape == (3, 7) fails immediately if a day is missing or a point was entered twice.
• The next calculations become one line (L22–L23): one point's week, every point's Saturday, totals and best days.

The business question always decides the layout. Rows should be the things you compare (pickup points). Columns should be the repeated measurements (days). If you choose the layout for no reason, later calculations become confusing.

---

13. Workplace Task — Set Up the Grid for the Weekly Review

Fictional business: Cedar Lane. Your manager writes:

"Starting next week, I want one notebook for the weekly pickup-point review instead of three spreadsheets. Can you set it up so that anyone can rerun it?"

Create a notebook L21_weekly_review_setup.ipynb that:

1. starts with a Markdown cell stating the purpose of the notebook, that the data is synthetic, and what rows and columns mean;
2. has a version cell (as in Section 7, Part 1);
3. builds the (3, 7) units array from the worked example;
4. includes assertions for the shape and for North's total (cross-checked against the original list);
5. ends with a Markdown cell of two sentences to your manager: what the notebook contains, and the one thing they must do before trusting a rerun (the restart-and-run-all check);
6. passes Restart Kernel and Run All Cells….

---

14. Mini Challenge — The Wrong Shape

Fictional business: Hilltop Honey (from U03). A colleague recorded jar sales for 2 market stalls over 5 days. They expected shape (2, 5), but their code prints (5, 2):

import numpy as np

sales = np.array([
    [14, 9],
    [11, 12],
    [16, 10],
    [13, 15],
    [20, 18],
])
print(sales.shape)

1. Read the data before changing anything. What do the rows of this array represent, and what do the columns represent?
2. Rewrite the array so that rows = stalls and columns = days, with shape (2, 5). Type the values in the new layout; do not use any function you have not learned yet.
3. Add an assert that checks the new shape. Also add one that checks that the sum of all values is unchanged. Hand-calculate that sum first.
4. In one sentence, explain why "the code runs" did not mean "the layout is right".

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 30 min, alongside setup):

• NumPy: Installing NumPy. https://numpy.org/install/. Read the pip section and the tip about environments. (Opened 15 Oct 2026.)
• Python docs: venv — creation of virtual environments. https://docs.python.org/3/library/venv.html. Use the activation table. (Opened 15 Oct 2026; the page showed the 3.15.0 documentation.)
• NumPy Quickstart (NumPy v2.5 manual), sections "The basics" and "Array creation". https://numpy.org/doc/stable/user/quickstart.html. (Opened 15 Oct 2026.)

Recommended:

• Project Jupyter: Installing Jupyter. https://jupyter.org/install
• JupyterLab 4.6.4: Notebooks. https://jupyterlab.readthedocs.io/en/stable/user/notebook.html
• Python Packaging User Guide: Installing Packages. https://packaging.python.org/en/latest/tutorials/installing-packages/

Reference (look things up when needed):

• JupyterLab 4.6.4: Commands list. Find "Restart Kernel and Run All Cells…". https://jupyterlab.readthedocs.io/en/stable/user/commands_list.html
• Python Developer's Guide: Status of Python versions. https://devguide.python.org/versions/
• Using Python on Windows and Using Python on macOS (Python docs). https://docs.python.org/3/using/windows.html · https://docs.python.org/3/using/mac.html

Go deeper (optional):

• Python for Data Analysis, 3rd ed., Ch. 4 "NumPy Basics", section 4.1 (the ndarray). https://wesmckinney.com/book/numpy-basics.html. Note that the book was written for pandas 2.0; the NumPy basics in section 4.1 match this lesson.

---

16. Person to Follow / Role Model

Fernando Pérez (mapped by D.World for L21).

• Verified facts (checked 15 Oct 2026):
• IPython's own history page says IPython was started in 2001 by Fernando Pérez while he was a graduate student.
• Project Jupyter's "About" page says Jupyter was "born out of the IPython Project in 2014".
• Why D.World recommends him: the notebook you set up today comes from that line of work. It started as one person's tool for interactive scientific computing and became an open project used across data science.
• Honest note: we cite only the two facts above. His current roles change over time; check the official IPython and Jupyter pages for details.
• Try this: open the IPython history page and find one problem IPython was created to solve. In one sentence, connect it to something you did in your notebook today.

---

17. Glossary

Term	Meaning in this lesson
Package	Code published for others to install with pip (e.g., NumPy)
Virtual environment (venv)	A separate folder with its own Python and packages, activated per terminal
pip	Python's package installer; use it as python -m pip ...
Jupyter notebook	A document of code and Markdown cells, saved as .ipynb
Cell	One block in a notebook: code or Markdown
Kernel	The Python process behind a notebook that runs code and remembers variables
Hidden state	Variables the kernel still remembers although the cells that made them were changed, reordered or deleted
import / alias	Loading a package (import numpy) / its short name (as np)
NumPy array (ndarray)	A grid of values that all share one dtype
Element	One value inside an array
Shape	Elements per dimension, as a tuple, e.g. (3, 7) = 3 rows, 7 columns
ndim	Number of dimensions (1 for a row of values, 2 for a grid)
size	Total number of elements (the product of the shape)
dtype	The single data type of every element, e.g. int64, float64, bool
Tuple	An unchangeable sequence in round brackets; (7,) has one item
Upcasting	NumPy choosing a wider dtype (e.g., float64) so all inputs fit

Review from Stage 1: list and indexing (L15) · int/float/bool (L13) · function call and error message (L16).

---

18. Reflection

1. Which setup step took longest, and what would you tell a classmate to make it faster?
2. Before running the worked example, did you predict the shape as (3, 7) or (7, 3)? What would each layout mean for Cedar Lane?
3. Describe hidden state in your own words. When could it mislead your manager?

---

19. Next Step

You can now build a grid and describe it precisely. In L22 — Indexing, Slicing & Boolean Masks, you will select from it: one pickup point's week, every point's Friday, the weekend block, and the cells below a sales threshold. Knowing the shape is what lets you predict what each selection returns. Keep dworld-env and your notebook. L22 uses the same environment, and nothing new needs to be installed.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Setup evidence: a notebook with a version cell that passes Restart Kernel and Run All Cells	A working Stage 2 environment (or a hosted Jupyter environment)	Rubric L21-SETUP (versions in range, clean rerun)
Knowledge check (Q1–Q8)	Predicts shape and dtype; distinguishes list repetition from element-wise multiplication; understands hidden state	7 or more of 8 correct
Guided practice (Parts 1–5)	Builds and inspects 1-D and 2-D arrays and cross-checks with plain Python	Rubric L21-GP
Independent practice, workplace task, mini challenge	Chooses and justifies a shape without hints; makes a notebook others can rerun	Rubrics L21-IP, L21-WT, L21-MC

This lesson's evidence contributes to U06. The U06 Exit Check comes after L24 and is not part of this lesson.

---

References

1. NumPy Developers. Installing NumPy. https://numpy.org/install/ (opened 15 Oct 2026). Supports: installing NumPy with pip, and the recommendation to use an environment.
2. NumPy Developers. NumPy 2.5.0 Release Notes. https://numpy.org/doc/stable/release/2.5.0-notes.html (opened 15 Oct 2026). Supports: "This release supports Python versions 3.12-3.14."
3. NumPy Developers. NumPy quickstart (v2.5 manual). https://numpy.org/doc/stable/user/quickstart.html (opened 15 Oct 2026). Supports: ndarray attributes ndim, shape, size, dtype; array, zeros, ones, arange.
4. Python Software Foundation. venv — Creation of virtual environments. https://docs.python.org/3/library/venv.html (opened 15 Oct 2026; 3.15.0 docs). Supports: python -m venv; the activation commands per shell; the PowerShell execution-policy note.
5. Python Software Foundation. Using Python on Windows. https://docs.python.org/3/using/windows.html (opened 15 Oct 2026; 3.15.0 docs). Supports: "To obtain Python from the CPython team, use the Python Install Manager"; py install; py -V:3.14; the full installer deprecated since 3.14; the optional PATH entry.
6. Python Software Foundation. Using Python on macOS. https://docs.python.org/3/using/mac.html (opened 15 Oct 2026; 3.15.0 docs). Supports: python.org .pkg installer; the /Applications/Python 3.x folder; Install Certificates.command; administrator account; do not modify Apple's /usr/bin/python3.
7. Python Software Foundation. Status of Python versions (Developer's Guide). https://devguide.python.org/versions/ (opened 15 Oct 2026, via the Stage 2 map log). Supports: 3.15 released 2026-10-09; 3.14 in bugfix status.
8. Python Packaging Authority. Installing Packages. https://packaging.python.org/en/latest/tutorials/installing-packages/ (via the Stage 2 map log). Supports: pip with virtual environments; the python -m pip form.
9. Project Jupyter. Installing Jupyter. https://jupyter.org/install (via the Stage 2 map log). Supports: pip install jupyterlab, then jupyter lab.
10. Project Jupyter. JupyterLab 4.6.4: Notebooks and Commands list. https://jupyterlab.readthedocs.io/en/stable/user/notebook.html · https://jupyterlab.readthedocs.io/en/stable/user/commands_list.html (opened 15 Oct 2026). Supports: creating a notebook via + and the Launcher with a kernel; command labels "Restart Kernel and Run All Cells…" and "Change Kernel…".
11. IPython. History. https://ipython.readthedocs.io/en/stable/about/history.html · Project Jupyter. About. https://jupyter.org/about (via the Stage 2 map log, IPY-HIST and JUP-ABOUT). Support the role-model facts.
12. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 4. https://wesmckinney.com/book/numpy-basics.html (via the Stage 2 map log, WM-CH4).
13. D.World Stage 2 Detailed Lesson Map v1.0 (L21 row and U06 setup requirement), Curriculum Map v1.0 and Curriculum Bible v1.0. Internal.

L21 — تجهيز Python وJupyter وNumPy: المصفوفات والأشكال وأنواع البيانات

ملف المتعلّم. الإجابات والحلول النموذجية في ملف منفصل خاص بالمدرّس فقط. الإصدار: U06 Batch L21–L23 v1 · تاريخ المحتوى: 15 أكتوبر 2026. تقديرات الوقت تقديرات للتخطيط إلى أن يختبرها المتعلّمون. الكود في هذا الدرس شُغّل باستخدام تجهيزات الدورة من L21 (Python 3.13 وNumPy 2.5) على Linux وعلى Windows 11 في 4 نوفمبر 2026؛ وعلى Windows 11 أُنشئت البيئة بسطر التثبيت الموجود في الجزء A (راجع تقرير اختبار الكود Code Test Report). ولم يُختبر التثبيت على macOS ولا على Python 3.14. إذا لم تنجح خطوة ما على جهازك، فاستخدم بديل الدعم في القسم 4.

كل الأنشطة التجارية خيالية، وكل البيانات مصطنعة.

---

1. بطاقة الدرس

البند	القيمة
العنوان	تجهيز Python وJupyter وNumPy: المصفوفات والأشكال وأنواع البيانات
المرحلة	المرحلة 2 — التعامل مع البيانات
الوحدة	U06 — أساسيات NumPy
رقم الدرس	L21 (أول درس في U06 وفي المرحلة 2)
المهارات	المصفوفات (arrays)؛ الأشكال (shapes)
المستوى	مبتدئ في NumPy (مع افتراض معرفة Python من المرحلة 1)
الوقت المتوقع	حوالي 3 ساعات (تقدير للتخطيط، غير مُتحقَّق منه). ويشمل ذلك حوالي 45–60 دقيقة لتجهيز يتم مرة واحدة. ووقت التجهيز يختلف كثيرًا حسب الجهاز ونظام التشغيل.
مستوى التقييم	الفهم ← التطبيق
المتطلبات السابقة	المرحلة 1. من U04: المتغيرات وint/float/bool/str (L13)، والمقارنات (L14)، والقوائم والحلقات التكرارية والفهرسة التي تبدأ من الصفر (L15)، والدوال ورسائل الخطأ وassert (L16). ومن U05: الإجابة المتوقعة أولًا، والتتبّع والاختبار (L19). ومن U02: الإيرادات والتكلفة ومجمل الربح والهامش (L06).
الجديد في هذا الدرس	import، والحزم (packages)، والبيئات الافتراضية، والدفاتر (notebooks)، والصفوف الثابتة (tuples)، والملفات والمجلدات في سطر الأوامر. ولا تحتاج إلى معرفتها مسبقًا.

تفصيل الوقت (تقدير للتخطيط):

• التجهيز (القسم 4، الجزء A): 45–60 دقيقة.
• أساسيات الدفتر: 15 دقيقة.
• المفهوم والمثال المحلول: 35 دقيقة.
• التدريب الموجَّه: 25 دقيقة.
• التدريب المستقل: 15 دقيقة.
• اختبار الفهم: 10 دقائق.
• الذكاء الاصطناعي في التطبيق والتحقق: 10 دقائق (اختياري).
• مهمة العمل والتحدي القصير: 20 دقيقة.

ما تحتاجه:

• جهاز كمبيوتر تستطيع تثبيت البرامج عليه، أو بيئة Jupyter مستضافة على الإنترنت (راجع "إذا لم ينجح التجهيز").
• اتصالًا بالإنترنت مرة واحدة، لتنزيل Python والحزم.
• لا حساب، ولا مفتاح API، ولا برامج مدفوعة. لا شيء في المرحلة 2 يتطلبها.

---

2. هدف التعلّم

في نهاية هذا الدرس ستكون قادرًا على أن تجهّز بيئة عمل للمرحلة 2، وأن تعرض إصداراتها في دفتر، وأن تنشئ مصفوفات NumPy أحادية البعد (1-D) وثنائية البعد (2-D) تتوقع شكلها (shape) وعدد أبعادها (ndim) وحجمها (size) ونوع بياناتها (dtype) بشكل صحيح قبل تشغيل الكود.

الهدف المعتمد من D.World (بنصّه): Set up a working virtual environment with Python 3.13-3.14, NumPy, pandas, openpyxl and JupyterLab, show the printed versions in a notebook, and create 1-D and 2-D NumPy arrays whose shape, ndim, size and dtype are predicted correctly before running the code.

---

3. لماذا يهمّك هذا؟

في U04 خزّنت مبيعات أسبوع في قائمة (list) في Python، واستخدمت حلقة تكرارية لجمعها. هذا ينفع لقائمة واحدة. لكن العمل الحقيقي نادرًا ما يبقى بهذا الصغر. أصبح لدى Cedar Lane، وهو متجر أدوات مكتبية خيالي، ثلاث نقاط استلام. وكل نقطة تحتفظ بقائمة خاصة بها للوحدات المباعة يوميًا. ولكي يقارن بينها، ينسخ شخص ما الأرقام بين القوائم يدويًا، فتتسلل الأخطاء.

NumPy يخزّن كل هذه الأرقام في شبكة واحدة تسمى المصفوفة (array). كل صف فيها نقطة استلام، وكل عمود يوم. وبمجرد أن تصبح البيانات في شبكة، يستطيع سطر كود واحد أن يجري حسابًا على كل القيم دفعة واحدة. وكل أدوات المرحلة 2 اللاحقة مبنية على هذه الفكرة. ففي U07، كل عمود في جدول pandas هو مصفوفة لها تسمية، على نمط NumPy.

سياق مهني. خريطة منهج D.World تذكر NumPy ضمن البرامج التي يتطلبها عمل عالِم البيانات حاليًا. وهذه خلفية توضّح سبب وجود المهارة في المنهج. وهي ليست وعدًا بأي وظيفة أو نتيجة.

قبل كل ذلك، تحتاج إلى مكان موثوق لتشغيل الكود. هذا الدرس يجهّز البيئة التي ستستخدمها المرحلة 2 كلها، بحيث لا تحتاج الدروس اللاحقة إلى تثبيت أي شيء جديد.

المبدأ الأساسي: توقّع الشكل قبل أن تشغّل الكود. إذا لم تستطع أن تقول كيف يجب أن تبدو الشبكة، فلن تستطيع أن تعرف هل بناها الكمبيوتر بشكل صحيح.

---

4. المفهوم

الجزء A — جهّز بيئة المرحلة 2 (مرة واحدة فقط)

المكوّنات ولماذا نحتاج كلًّا منها

المكوّن	ما هو	لماذا تحتاجه المرحلة 2
Python 3.14 (ويُقبل 3.13)	اللغة التي استخدمتها في U04	NumPy 2.5 يدعم Python من 3.12 إلى 3.14. وتستخدم D.World الإصدار 3.14 في التجهيزات الجديدة وتقبل 3.13.
الحزمة (Package)	كود إضافي كتبه شخص آخر ونشره، ويُثبَّت باستخدام pip	NumPy وpandas وopenpyxl حزم. وهي ليست جزءًا من Python نفسه.
البيئة الافتراضية (Virtual environment) ("venv")	مجلد مستقل له Python خاص به وحزم خاصة به	تجمع إصدارات المرحلة 2 معًا وتمنع تعارضها مع مشروعات أخرى.
NumPy 2.5.x	المصفوفات والعمل العددي السريع	U06
pandas 3.0.x	الجداول (DataFrames)	من U07 فصاعدًا. ونثبّته الآن حتى لا تحتاج الدروس اللاحقة إلى تثبيت أي شيء جديد.
openpyxl 3.1.5	يتيح لـ pandas قراءة ملفات Excel ‏.xlsx والكتابة فيها	U07 وU08
JupyterLab 4.6.x	محرّر يعمل في المتصفح لكتابة الدفاتر (notebooks)	أعمال المرحلة 2 تُسلَّم في صورة دفاتر. ويُقبل أيضًا Notebook الكلاسيكي 7.6.x.

لا تستخدم Python 3.15 في المرحلة 2 بعد، حتى لو كان قد صدر في 9 أكتوبر 2026. ففي 15 أكتوبر 2026، كانت ملاحظات إصدار NumPy 2.5 تذكر دعمًا حتى Python 3.14 فقط. ولا تستخدم 3.12 أو أقدم في تجهيز جديد أيضًا. إذا كنت قد ثبّت إصدارًا أقدم من Python في L13، فاحتفظ به، وأضف 3.14 بجانبه.

حرف "x" في رقم الإصدار يعني "أي رقم هنا". مثلًا، 2.5.x تعني 2.5.0 أو 2.5.1 أو 2.5.4 وهكذا. والأمر الموجود بالأسفل يطلب من pip أحدث إصدار في هذه السلسلة.

الخطوة 1 — ثبّت Python 3.14

افتح الطرفية (terminal): Terminal في macOS وLinux، وPowerShell أو Command Prompt في Windows. الطرفية نافذة تكتب فيها الأوامر وتضغط Enter.

Windows. يقول توثيق Python الخاص بـ Windows: "To obtain Python from the CPython team, use the Python Install Manager." (للحصول على Python من فريق CPython، استخدم Python Install Manager). نزّله من python.org/downloads أو من Microsoft Store. ثم اكتب في PowerShell:

py install 3.14
py -V:3.14 --version

• الأمر الثاني يجب أن يطبع Python 3.14. متبوعًا برقم.
• إذا عرض برنامج التثبيت إضافة مجلد إلى PATH، فيمكنك أن توافق. وهذا اختياري، لأن الأمر py يعمل بدونه.
• يذكر توثيق Python أن برنامج التثبيت الكامل القديم أصبح مهجورًا (deprecated) بدءًا من 3.14. استخدم مدير التثبيت ما لم يطلب منك دعم D.World غير ذلك.

macOS. نزّل حزمة التثبيت الخاصة بـ macOS (.pkg) لإصدار Python 3.14 من python.org/downloads/macos. ثم:

1. انقر نقرًا مزدوجًا على الحزمة واتبع خطوات التثبيت. وتحتاج إلى حساب مسؤول (administrator).
2. بعد ذلك، افتح المجلد /Applications/Python 3.14/. وانقر نقرًا مزدوجًا على Install Certificates.command. فبدون هذه الخطوة، قد تفشل التنزيلات من Python بأخطاء في الشهادات.
3. في Terminal، تأكد:

python3.14 --version

لا تغيّر ولا تحذف /usr/bin/python3 الخاص بـ Apple.

Linux. تأكد أولًا مما لديك بالفعل:

python3 --version
python3.14 --version

• إذا عرض أيٌّ من الأمرين 3.14 أو 3.13، فاستخدم ذلك الأمر في الخطوة 2.
• وإذا لم يحدث ذلك، فثبّت Python 3.14 أو 3.13 باستخدام مدير الحزم في توزيعتك، إذا كان يوفّر أحدهما. وإلا فاستخدم البديل الموجود في "إذا لم ينجح التجهيز" بالأسفل.
• في Debian وUbuntu، قد تكون الوحدة venv حزمة منفصلة. إذا ظهرت في الخطوة 2 رسالة ensurepip is not available، فثبّت الحزمة المذكورة في تلك الرسالة (مثل python3-venv)، ثم كرّر الخطوة 2.

الخطوة 2 — أنشئ مجلد مشروع وبيئة افتراضية

أنشئ مجلدًا للمرحلة 2، مثل dworld-stage2، وادخل إليه. ثم أنشئ بيئة اسمها dworld-env:

النظام	الأوامر
Windows (PowerShell)	mkdir dworld-stage2 · cd dworld-stage2 · py -V:3.14 -m venv dworld-env
macOS	mkdir dworld-stage2 · cd dworld-stage2 · python3.14 -m venv dworld-env
Linux	mkdir dworld-stage2 · cd dworld-stage2 · python3.14 -m venv dworld-env (أو python3.13 ...)

لا يُطبع شيء عندما ينجح الأمر. ويظهر مجلد جديد اسمه dworld-env.

الخطوة 3 — فعّل البيئة

التفعيل يعني: "استخدم Python وpip الخاصين بهذه البيئة في نافذة الطرفية هذه". ويجب أن تفعل ذلك في كل مرة تفتح فيها طرفية جديدة للعمل على المرحلة 2.

النظام / الواجهة	الأمر
macOS أو Linux (bash/zsh)	source dworld-env/bin/activate
Windows Command Prompt	dworld-env\Scripts\activate.bat
Windows PowerShell	dworld-env\Scripts\Activate.ps1

عندما ينجح التفعيل، يبدأ سطر الأوامر عادةً بـ (dworld-env).

هل منع PowerShell تشغيل السكربت؟ قد يرفض Windows تشغيل Activate.ps1. ويصف توثيق Python إعدادًا اسمه "execution policy" (سياسة التنفيذ) يتحكم في ذلك. لا تغيّر إعدادات الأمان على جهاز مشترك أو جهاز عمل دون إذن. وأبسط حل هو استخدام Command Prompt وactivate.bat بدلًا من ذلك. ويمكنك أيضًا أن تسأل دعم D.World.

الخطوة 4 — ثبّت الحزم

والبيئة مفعّلة، اكتب هذا في سطر واحد:

python -m pip install "numpy==2.5.*" "pandas==3.0.*" "openpyxl==3.1.5" "jupyterlab==4.6.*"

• السطر نفسه يعمل على Windows وmacOS وLinux بعد تفعيل البيئة.
• استخدم python -m pip بدلًا من pip وحدها. فهذا يضمن أن الحزم تُثبَّت في Python الذي تستخدمه فعلًا.
• بديل: ملفات التدريب تحتوي على requirements-stage2.txt، وفيه الأسطر الأربعة نفسها. ثبّت منه باستخدام python -m pip install -r requirements-stage2.txt.
• التثبيت ينزّل ملفات وقد يستغرق عدة دقائق.

الخطوة 5 — شغّل JupyterLab وأنشئ دفترًا

والبيئة ما زالت مفعّلة:

jupyter lab

1. يفتح متصفحك JupyterLab. اترك نافذة الطرفية مفتوحة أثناء العمل؛ فإغلاقها يوقف JupyterLab.
2. في JupyterLab، اضغط زر + في متصفح الملفات. فيُفتح تبويب Launcher.
3. تحت Notebook، اختر نواة (kernel) Python 3 (وعادةً تكون تسميتها Python 3 (ipykernel)). ولأنك شغّلت JupyterLab من البيئة المفعّلة، فهذه النواة تستخدم dworld-env.
4. غيّر اسم الدفتر إلى L21_setup_and_arrays.ipynb: انقر بزر الفأرة الأيمن على تبويبه أو على اسمه في متصفح الملفات.
5. لإيقاف JupyterLab لاحقًا، احفظ عملك، ثم اضغط Ctrl + C في الطرفية وأكّد.

في المرة القادمة: افتح طرفية ← ادخل إلى dworld-stage2 ← فعّل البيئة ← jupyter lab. لا تحتاج إلى التثبيت مرة أخرى.

إذا لم ينجح التجهيز — بديل الدعم

مشكلات التجهيز شائعة، وليست علامة على أنك لا تستطيع تعلّم علم البيانات. إذا علقت أكثر من 30 دقيقة تقريبًا:

1. انسخ السطر الأخير من رسالة الخطأ.
2. احذف منه اسم المستخدم وأسماء المجلدات (راجع القسم 10).
3. أرسله إلى دعم D.World.

سيساعدك دعم D.World (support.dworld@gmail.com أو WhatsApp، والمذكوران في صفحة الدعم) على إصلاح التجهيز انطلاقًا من ذلك السطر. وإذا لم يستطع جهازك تشغيل التجهيز، فيمكنك استخدام أي خدمة Jupyter مستضافة متاحة لك، وتشغيل سطر التثبيت من الجزء A في أول خلية فيها، ثم التأكد من الإصدارات بخلية الإصدارات. واستخدامها مقبول تمامًا في المرحلة 2 كلها. وإذا لم تستطع استخدام Jupyter على الإطلاق، فسيسجّل دعم D.World استثناءً لك. وعندها يُقبل سكربت Python لأعمال U06–U08. أما دفتر المشروع 2 فسيُنجز في Jupyter مع ذلك (على جهازك أو في خدمة مستضافة).

الجزء B — أساسيات الدفتر

الدفتر (notebook) مستند مكوّن من خلايا (cells):

نوع الخلية	تحتوي على	تشغّلها باستخدام
خلية كود (Code cell)	كود Python. وتظهر مخرجاتها تحتها مباشرة.	Shift + Enter (يشغّل الخلية وينتقل إلى التالية)
خلية Markdown	ملاحظات وعناوين بنص عادي	Shift + Enter يعرضها منسّقة

خلف كل دفتر تعمل نواة (kernel): وهي عملية Python التي تنفّذ الكود وتتذكر كل متغير أنشأته. وهذه الذاكرة هي سبب أشهر خطأ في الدفاتر.

الحالة المخفية (Hidden state). افترض أنك شغّلت خلية تجعل price = 12، ثم حذفت تلك الخلية. النواة ما زالت تتذكر price، ولذلك تستمر الخلايا اللاحقة في العمل. لكن عندما يفتح شخص آخر دفترك، لن يكون price موجودًا، وسيفشل التشغيل عنده. وتشغيل الخلايا بترتيب خاطئ يسبب المشكلة نفسها. والرقم الموجود بين [ ] بجانب كل خلية يوضّح الترتيب الذي شُغّلت به الخلايا فعلًا.

الحل، وقاعدة من قواعد المرحلة 2. قبل أن تسلّم أي دفتر أو تشاركه، استخدم Kernel → Restart Kernel and Run All Cells…. فهذا يمسح ذاكرة النواة ويشغّل كل الخلايا من أعلى إلى أسفل. وإذا انتهى دون خطأ، فالدفتر يُعاد إنتاجه من الصفر. كل دفتر يُسلَّم في المرحلة 2 يجب أن يجتاز هذا الفحص.

في الدفتر، إذا كان السطر الأخير من الخلية تعبيرًا، يعرض Jupyter قيمته حتى دون print(). وتظهر المصفوفات عندئذ بشكل array([12, 15, ...]). ويستخدم هذا الدرس print() حتى تبدو المخرجات متطابقة في الدفتر وفي السكربت.

الجزء C — استيراد NumPy

في U04 كان كل ما استخدمته مدمجًا في Python. أما الحزمة فيجب استيرادها (import) قبل استخدامها:

import numpy as np

• import numpy تحمّل الحزمة.
• as np تعطيها اسمًا مختصرًا (alias). وكتابة np بدلًا من numpy عُرف شبه عالمي؛ وستراه في التوثيق وفي كود الآخرين.
• بعد هذا السطر، تُكتب أدوات NumPy بصيغة np.something، مثل np.array.
• شغّل خلية الاستيراد أولًا. وإذا ظهرت لك NameError: name 'np' is not defined، فخلية الاستيراد لم تُشغَّل بعد في هذه النواة (الحالة المخفية مرة أخرى).

الجزء D — الصفوف الثابتة (Tuples) باختصار

الصف الثابت (tuple) يشبه القائمة لكنه يُكتب بين أقواس دائرية: (3, 7). لن تنشئ صفوفًا ثابتة بنفسك في U06. لكنك تحتاج إلى قراءتها، لأن NumPy يعرض شكل (shape) المصفوفة في صورة صف ثابت.

المكتوب	المعنى
(3, 7)	رقمان: 3 صفوف و7 أعمدة
(7,)	رقم واحد: 7. والفاصلة في النهاية هي التي تجعله صفًا ثابتًا طوله 1.
()	لا أبعاد. ستراه مع القيمة المفردة.

وعلى عكس القائمة، لا يمكن تغيير الصف الثابت بعد إنشائه.

الجزء E — من القائمة إلى المصفوفة

import numpy as np

north_list = [12, 15, 11, 14, 18, 25, 22]   # Mon..Sun units, North point (synthetic)
north = np.array(north_list)

print(north_list)
print(north)
print(type(north))

[12, 15, 11, 14, 18, 25, 22]
[12 15 11 14 18 25 22]
<class 'numpy.ndarray'>

(التعليق يعني: الوحدات من الإثنين إلى الأحد، لنقطة North، بيانات مصطنعة.) عندما تطبع مصفوفة، لا تظهر فيها فواصل. ونوعها numpy.ndarray، وهو اختصار لـ "N-dimensional array" (مصفوفة متعددة الأبعاد).

القائمة مقابل المصفوفة: الفرقان المهمان الآن

	قائمة Python	مصفوفة NumPy
القيم	يمكن أن تخلط الأنواع: [12, "Mon", True]	نوع بيانات (dtype) واحد لكل العناصر
* 2	تكرّر القائمة	تضرب كل عنصر (عنصرًا بعنصر، element-wise)

print(north_list * 2)
print(north * 2)

[12, 15, 11, 14, 18, 25, 22, 12, 15, 11, 14, 18, 25, 22]
[24 30 22 28 36 50 44]

الرمز نفسه يعني شيئين مختلفين تمامًا، ولذلك اعرف دائمًا أي نوع من الكائنات بين يديك.

الجزء F — الشكل وعدد الأبعاد والحجم ونوع البيانات

لكل مصفوفة أربع خصائص (attributes) ستراجعها باستمرار. ولاحظ أنه لا توجد أقواس بعدها: فهي خصائص، وليست استدعاءات لدوال.

الخاصية	السؤال الذي تجيب عنه	north
.shape	كم عنصرًا على امتداد كل بُعد؟ (صف ثابت)	(7,)
.ndim	كم بُعدًا؟	1
.size	كم عنصرًا إجمالًا؟	7
.dtype	ما نوع كل عنصر؟	int64

print(north.shape, north.ndim, north.size, north.dtype)

(7,) 1 7 int64

مصفوفة ثنائية البعد: صفوف وأعمدة لها معنى في الأعمال. القائمة المكوّنة من قوائم، كلها بالطول نفسه، تصبح شبكة. وهنا الصفوف = نقاط الاستلام (North وCentral وHarbor) والأعمدة = الأيام (من الإثنين … إلى الأحد):

units = np.array([
    [12, 15, 11, 14, 18, 25, 22],   # North
    [20, 18, 22, 19, 31, 30, 28],   # Central
    [ 8,  0,  9, 10, 12, 16, 17],   # Harbor
])
print(units)
print(units.shape, units.ndim, units.size, units.dtype)

[[12 15 11 14 18 25 22]
 [20 18 22 19 31 30 28]
 [ 8  0  9 10 12 16 17]]
(3, 7) 2 21 int64

اقرأ الشكل بترتيب (الصفوف، الأعمدة): (3, 7) تعني 3 صفوف (نقاط استلام) و7 أعمدة (أيام). والترتيب مهم. فالشكل (7, 3) سيعني 7 صفوف و3 أعمدة، أي ترتيبًا مختلفًا للأرقام الـ 21 نفسها. والحجم size هو دائمًا حاصل ضرب أرقام الشكل: 3 × 7 = 21.

أنواع البيانات (dtypes) التي ستقابلها في U06

dtype	يحمل	يُنشأ من
int64	أعداد صحيحة	مُدخلات كلها int
float64	أعداد عشرية	وجود أي float في المُدخلات، وnp.zeros، وnp.ones
bool	True / False	مُدخلات كلها bool، أو مقارنة (L22)

الرقم 64 يعني أن كل قيمة تستخدم 64 بت (bits) من الذاكرة. ومع NumPy 2.x، نوع الأعداد الصحيحة الافتراضي هو int64 على Windows وmacOS وLinux على حد سواء.

الترقية (Upcasting). لأن المصفوفة لها نوع بيانات واحد، يختار NumPy نوعًا يتسع لكل القيم المُدخلة. خلط الأعداد الصحيحة بالعشرية يعطي float64، وتُرقّى الأعداد الصحيحة إلى عشرية. وخلط bool مع int يعطي int64 (حيث تصبح True هي 1، وFalse هي 0).

mixed = np.array([3, 4.5, 6])
print(mixed, mixed.dtype)
print(np.array([1, True, 0]), np.array([1, True, 0]).dtype)
print(np.array([True, False]).dtype)

[3.  4.5 6. ] float64
[1 1 0] int64
bool

3. تعني 3.0. ويضيف NumPy مسافات إلى الأرقام القصيرة حتى تصطف الأعمدة.

انتبه للنصوص. np.array(["12", "15"]) تنشئ مصفوفة من النصوص (ويبدأ نوع بياناتها بـ <U)، وليست أرقامًا. والعمليات الحسابية عليها تفشل. والبيانات التي تصل كنص يجب تحويلها أولًا. فعلت ذلك مع القيم المفردة في L13؛ وستفعله U08 وU09 مع أعمدة كاملة.

الجزء G — إنشاء المصفوفات دون كتابة كل قيمة

الدالة	تنشئ	مثال	النتيجة
np.arange(n)	0، 1، …، n − 1 (مثل range)	np.arange(7)	[0 1 2 3 4 5 6]
np.arange(a, b)	a، a + 1، …، b − 1	np.arange(1, 8)	[1 2 3 4 5 6 7]
np.zeros(shape)	كلها أصفار، float64	np.zeros((2, 3))	شبكة 2 × 3 من 0.
np.ones(shape)	كلها آحاد، float64	np.ones(4)	[1. 1. 1. 1.]

مع np.zeros وnp.ones، يوضع الشكل ثنائي البعد داخل أقواس خاصة به كصف ثابت: np.zeros((2, 3)). أما كتابة np.zeros(2, 3) فخطأ، لأن الرقم الثاني يُقرأ كإعداد مختلف.

day_numbers = np.arange(1, 8)
placeholder = np.zeros((2, 3))
print(day_numbers, day_numbers.dtype)
print(placeholder)
print(placeholder.shape, placeholder.dtype)

[1 2 3 4 5 6 7] int64
[[0. 0. 0.]
 [0. 0. 0.]]
(2, 3) float64

المصفوفة المؤقتة (placeholder) مفيدة عندما تعرف الترتيب قبل أن تتوفر البيانات، مثل شبكة ستُملأ كلما أُغلق حساب شهر.

---

5. مثال محلول

Cedar Lane (خيالي)، بيانات مصطنعة. في الأسبوع الماضي، باعت ثلاث نقاط استلام كشاكيل. ويريد المدير كل الأرقام في شبكة واحدة، مع فحص سريع يؤكد أن شيئًا لم يضِع عند دمج القوائم.

الخطوة 1 — الإجابة المتوقعة أولًا (قبل أي كود).

• 3 نقاط استلام × 7 أيام ← الشكل (3, 7)، وndim يساوي 2، وsize يساوي 21.
• كل القيم أعداد صحيحة ← نوع البيانات int64.
• إجمالي North يدويًا: 12 + 15 + 11 + 14 + 18 + 25 + 22 = 117.

الخطوة 2 — ابنِ وافحص.

import numpy as np

north_list   = [12, 15, 11, 14, 18, 25, 22]
central_list = [20, 18, 22, 19, 31, 30, 28]
harbor_list  = [ 8,  0,  9, 10, 12, 16, 17]

units = np.array([north_list, central_list, harbor_list])

print("shape:", units.shape)
print("ndim: ", units.ndim)
print("size: ", units.size)
print("dtype:", units.dtype)

shape: (3, 7)
ndim:  2
size:  21
dtype: int64

الخطوة 3 — قارن بالتوقع. القيم الأربع كلها مطابقة. ✔

الخطوة 4 — راجع صفًا واحدًا باستخدام Python العادي. units[0] يعطي الصف الأول، أي North. والفهرسة هي موضوع L22؛ ونستخدمها هنا فقط لمراجعة صف واحد. و.sum() تجمع عناصر المصفوفة؛ والتجميع (aggregation) موضوع L23.

print(sum(north_list))
print(units[0].sum())
assert units[0].sum() == sum(north_list)
assert units.shape == (3, 7)
print("checks passed")

117
117
checks passed

assert من L16: توقف البرنامج بخطأ AssertionError إذا كان الشرط خاطئًا، ولا تفعل شيئًا إذا كان صحيحًا. ووضع فحوصات الشكل بـ assert في دفترك يعني أن أي ترتيب خاطئ سيُكتشف فورًا.

---

6. شرح بصري

                 columns = days  (axis 1, length 7)
               Mon Tue Wed Thu Fri Sat Sun
             ┌───────────────────────────┐
  row 0 North│ 12  15  11  14  18  25  22│
  row 1 Centr│ 20  18  22  19  31  30  28│   rows = pickup points
  row 2 Harbr│  8   0   9  10  12  16  17│   (axis 0, length 3)
             └───────────────────────────┘
   shape = (3, 7)    ndim = 2    size = 3 × 7 = 21    dtype = int64

   list  [12, 15] * 2  ──►  [12, 15, 12, 15]   (repeats)
   array [12  15] * 2  ──►  [24  30]           (element-wise)

وصف الرسم: شبكة من 3 صفوف و7 أعمدة. الصفوف مسمّاة North وCentral وHarbor (نقاط الاستلام، المحور axis 0، وطوله 3). والأعمدة مسمّاة من الإثنين إلى الأحد (الأيام، المحور axis 1، وطوله 7). والقيم هي مبيعات الوحدات الأسبوعية من المثال المحلول. وتحت الشبكة: الشكل (3, 7)، وndim يساوي 2، وsize يساوي 21، وdtype هو int64. وفي الأسفل مقارنتان: القائمة [12, 15] مضروبة في 2 تكرّر القائمة، فتعطي [12, 15, 12, 15]. أما المصفوفة [12 15] مضروبة في 2 فتضرب كل عنصر، فتعطي [24 30].

---

7. تدريب موجَّه

اعمل في دفترك L21_setup_and_arrays.ipynb، أو افتح دفتر البداية L21_starter.ipynb من ملفات التدريب.

الجزء 1 — خلية الإصدارات. في أول خلية كود، اكتب وشغّل:

import sys
import numpy as np
import pandas as pd
import openpyxl
import jupyterlab

print("Python    ", sys.version.split()[0])
print("numpy     ", np.__version__)
print("pandas    ", pd.__version__)
print("openpyxl  ", openpyxl.__version__)
print("jupyterlab", jupyterlab.__version__)

assert sys.version_info[:2] in [(3, 13), (3, 14)], "Use Python 3.13 or 3.14"
assert np.__version__.startswith("2.5."), "Expected numpy 2.5.x"
assert pd.__version__.startswith("3.0."), "Expected pandas 3.0.x"
print("Environment OK")

يجب أن تبدو مخرجاتك هكذا. وقد تختلف الأرقام بعد النقطة الأخيرة عندك:

Python     3.14.x
numpy      2.5.x
pandas     3.0.x
openpyxl   3.1.5
jupyterlab 4.6.x
Environment OK

• إذا فشل أحد أسطر import برسالة ModuleNotFoundError، فالدفتر لا يستخدم dworld-env. وغالبًا يكون السبب أن JupyterLab شُغّل دون تفعيل البيئة أولًا.
• إذا فشلت إحدى جمل assert، فاقرأ رسالتها. فهي تخبرك أي إصدار خاطئ.

الجزء 2 — من القائمة إلى المصفوفة. حوّل قائمة North ذات الأيام السبعة إلى مصفوفة.

1. توقّع أولًا: اكتب shape وndim وsize وdtype الخاصة بها في خلية Markdown.
2. أنشئها، واطبع الخصائص الأربع، وقارنها بتوقعك.
3. راجع: قارن north.sum() بدالة Python ‏sum(north_list).

الجزء 3 — الشبكة 3 × 7. ابنِ مصفوفة units من المثال المحلول.

1. توقّع الخصائص الأربع، ثم اطبعها.
2. في خلية Markdown، اكتب بكلماتك ماذا يعني صف واحد وماذا يعني عمود واحد.

الجزء 4 — القائمة × 2 مقابل المصفوفة × 2. توقّع، ثم شغّل north_list * 2 وnorth * 2. واشرح الفرق في جملة واحدة.

الجزء 5 — إعادة التشغيل وتشغيل الكل. استخدم Kernel → Restart Kernel and Run All Cells…. يجب أن تعمل كل خلية من أعلى إلى أسفل دون خطأ. وإذا فشلت إحداها، فأصلح ترتيب خلاياك وكرّر. واحفظ الدفتر؛ فهو دليل التجهيز الخاص بك (القسم 20).

---

8. تدريب مستقل

الموقف (بيانات مصطنعة). يريد Cedar Lane متابعة الوحدات المباعة شهريًا لـ 4 منتجات (الكشاكيل، والأجندات، وأطقم الأقلام، والأوراق اللاصقة) على مدى 12 شهرًا. المبيعات لم تحدث بعد، ولذلك ستنشئ شبكة مؤقتة من الأصفار.

1. دون تلميحات، اختر الشكل. أنشئ الشبكة باستخدام np.zeros، ثم اطبع shape وndim وsize وdtype الخاصة بها.
2. في خلية Markdown، اذكر ماذا يعني صف واحد وماذا يعني عمود واحد في شبكتك.
3. اشرح في جملة واحدة لماذا اخترت هذا الترتيب وليس الآخر. الترتيبان ممكنان؛ والمهم هو السبب الذي تذكره.
4. توقّع نوع بيانات np.array([120, 45.5, 80, 30]). اكتب توقعك، ثم شغّله لتتأكد. واشرح لماذا يكون له هذا النوع.
5. توقّع حجم (size) شبكتك المؤقتة قبل أن تطبعه، ووضّح كيف حسبته.

شغّل Restart Kernel and Run All Cells… قبل أن تنتهي.

لا تُعرض هنا إجابة نموذجية.

---

9. اختبار الفهم (Knowledge Check) — توقّع الشكل ونوع البيانات

اكتب كل توقّع قبل أن تشغّل أي شيء. ثم شغّل الكود لتتأكد.

Q1. ماذا يطبع هذا الكود؟

a = np.array([[5, 1, 0, 2, 7, 3, 4],
              [2, 2, 6, 1, 0, 9, 8],
              [4, 4, 4, 4, 4, 4, 4]])
print(a.shape)

• A) (7, 3)
• B) (3, 7)
• C) 21
• D) (3,)

Q2. شبكة فيها الصفوف = 5 متاجر والأعمدة = 30 يومًا. أي شكل يطابقها؟

• A) (30, 5)
• B) (5, 30)
• C) (150,)
• D) (5,)

Q3. ماذا يطبع هذا الكود؟

weekly = [3, 4]
print(weekly * 2)

• A) [6, 8]
• B) [6 8]
• C) [3, 4, 3, 4]
• D) خطأ

Q4. ماذا يطبع هذا الكود؟

weekly = np.array([3, 4])
print(weekly * 2)

• A) [6 8]
• B) [3 4 3 4]
• C) [3, 4, 3, 4]
• D) خطأ

Q5. ما نوع بيانات (dtype) المصفوفة np.array([2, 7, 1.5])؟

• A) int64
• B) float64
• C) bool
• D) خليط من int64 وfloat64

Q6. ماذا يطبع هذا الكود؟

p = np.zeros((5, 6))
print(p.ndim, p.size, p.dtype)

• A) 2 30 float64
• B) 2 11 int64
• C) 6 30 float64
• D) 2 30 int64

Q7. حذفت خلية كانت تعرّف price، ومع ذلك ما زالت الخلايا التي تحتها تعمل. ثم فتح زميلك دفترك فظهرت له NameError: name 'price' is not defined. ما أفضل تفسير وحل؟

• A) لدى زميلك إصدار خاطئ من NumPy؛ ويجب أن يعيد التثبيت.
• B) نواتك كانت ما زالت تتذكر price (الحالة المخفية)؛ استخدم Restart Kernel and Run All Cells قبل المشاركة، وأعِد الخلية الناقصة.
• C) الدفاتر لا تستطيع مشاركة المتغيرات؛ انتقل إلى سكربت.
• D) price كلمة محجوزة في NumPy.

Q8. ما شكل np.array([10, 20, 30, 40, 50])؟

• A) (5,)
• B) (1, 5)
• C) (5, 1)
• D) 5

---

10. الذكاء الاصطناعي في التطبيق (اختياري، وفي حدود)

الهدف: الحصول على شرح بلغة بسيطة لخطأ في التجهيز، أو للفرق بين القائمة والمصفوفة، ثم التحقق منه بنفسك.

الخيار A — خطأ في التثبيت أو الاستيراد.

1. انسخ السطر الأخير من رسالة الخطأ، والأمر الذي كتبته.
2. احذف البيانات الشخصية قبل أن تلصق أي شيء (L12 في U03): اسم المستخدم، ومسارات المجلدات مثل C:\Users\amira\... أو /Users/amira/...، واسم جهازك، وأي عنوان بريد إلكتروني. واستبدلها بـ <USER> أو <PATH>.
3. اسأل:

"I am a beginner setting up Python 3.14 in a virtual environment on <Windows/macOS/Linux>. I ran this command: <command>. The last line of the error was: <error with personal details removed>. Explain in plain English what this error usually means and give me the two most likely causes. Do not tell me to change security settings or to install anything other than numpy, pandas, openpyxl or jupyterlab."

وبالعربية: "أنا مبتدئ أجهّز Python 3.14 في بيئة افتراضية على <Windows/macOS/Linux>. شغّلت هذا الأمر: <command>. وكان السطر الأخير من الخطأ: <الخطأ بعد حذف البيانات الشخصية>. اشرح بلغة بسيطة ماذا يعني هذا الخطأ عادةً، وأعطني السببين الأكثر احتمالًا. لا تطلب مني تغيير إعدادات الأمان، ولا تثبيت أي شيء غير numpy أو pandas أو openpyxl أو jupyterlab."

الخيار B — القائمة مقابل المصفوفة. اسأل:

"In Python, why does [3, 4] * 2 give a different result from numpy.array([3, 4]) * 2? Explain for a beginner, with one example of when each behaviour is useful."

وبالعربية: "في Python، لماذا يعطي [3, 4] * 2 نتيجة مختلفة عن numpy.array([3, 4]) * 2؟ اشرح لمبتدئ، مع مثال واحد على متى يكون كل سلوك منهما مفيدًا."

الحدود:

• الذكاء الاصطناعي اختياري. وكل مهمة في هذا الدرس يمكن إنجازها بدونه.
• استخدم فقط بيانات مصطنعة، أو رسائل الخطأ الخاصة بك بعد حذف البيانات الشخصية منها، أو الأمثلة الموجودة في هذا الدرس.
• لا تلصق كلمات مرور أو مفاتيح أو بيانات عملاء حقيقية.
• لا تشغّل أمرًا يقترحه الذكاء الاصطناعي حتى تفهم ما يفعله (القسم 11).

---

11. التحقق من مخرجات الذكاء الاصطناعي

1. هل يطابق ما رأيته؟ يجب أن يذكر الشرح نوع الخطأ نفسه والأمر نفسه. وإذا وصف خطأً مختلفًا، فهو يخمّن.
2. راجع الإصدارات، لا الآراء. خلية الإصدارات هي الدليل. وبعد تطبيق أي إصلاح مقترح، أعِد تشغيلها. يجب أن تطبع Python 3.13 أو 3.14، وnumpy 2.5.x، وpandas 3.0.x، وopenpyxl 3.1.5، وjupyterlab 4.6.x، ثم Environment OK.
3. ارفض النصائح الخطرة أو الخارجة عن النطاق. أمثلة: تغيير إعدادات الأمان أو سياسة التنفيذ على جهاز مشترك؛ أو التثبيت بصلاحيات المسؤول بينما يجب استخدام بيئة افتراضية؛ أو تثبيت إصدار مختلف من Python (مثل 3.15) أو حزم إضافية؛ أو استخدام أداة غير التي يستخدمها هذا الدرس. واسأل دعم D.World بدلًا من ذلك.
4. اختبر ادعاء الفرق بين القائمة والمصفوفة بنفسك. شغّل [3, 4] * 2 وnp.array([3, 4]) * 2. فالمخرجات، لا صياغة الذكاء الاصطناعي، هي ما تثق فيه.
5. استخدم Restart Kernel and Run All Cells بعد أي تغيير. فالإصلاح الذي يعمل فقط في نواتك الحالية ليس إصلاحًا.

---

12. منظور الأعمال

شبكة واحدة تجعل المراجعات الأسبوعية أسرع. كانت نقاط الاستلام الثلاث في Cedar Lane تحتفظ بالمبيعات اليومية في قوائم منفصلة. وهذا يعني ثلاثة ملفات يجب فتحها، وثلاثة مجاميع يجب حسابها، وثلاث فرص لنسخ رقم بشكل خاطئ.

مع شبكة واحدة شكلها (3, 7):

• الترتيب صريح. أي شخص يقرأ shape (3, 7) مع الملاحظة "الصفوف = نقاط الاستلام، والأعمدة = الأيام" يعرف بالضبط ما يمثّله كل رقم.
• الترتيب الخاطئ يُكتشف مبكرًا. فالجملة assert units.shape == (3, 7) تفشل فورًا إذا نقص يوم أو أُدخلت نقطة مرتين.
• الحسابات التالية تصبح سطرًا واحدًا (L22–L23): أسبوع نقطة واحدة، ويوم السبت لكل النقاط، والمجاميع، وأفضل الأيام.

سؤال الأعمال هو الذي يحدد الترتيب دائمًا. فالصفوف يجب أن تكون الأشياء التي تقارن بينها (نقاط الاستلام). والأعمدة يجب أن تكون القياسات المتكررة (الأيام). وإذا اخترت الترتيب دون سبب، فستصبح الحسابات اللاحقة مربكة.

---

13. مهمة عمل — جهّز الشبكة للمراجعة الأسبوعية

نشاط تجاري خيالي: Cedar Lane. يكتب مديرك:

"بدءًا من الأسبوع القادم، أريد دفترًا واحدًا لمراجعة نقاط الاستلام الأسبوعية بدلًا من ثلاثة جداول بيانات. هل تستطيع أن تجهّزه بحيث يستطيع أي شخص أن يعيد تشغيله؟"

أنشئ دفترًا باسم L21_weekly_review_setup.ipynb:

1. يبدأ بخلية Markdown تذكر غرض الدفتر، وأن البيانات مصطنعة، وماذا تعني الصفوف والأعمدة؛
2. فيه خلية إصدارات (كما في القسم 7، الجزء 1)؛
3. يبني مصفوفة units ذات الشكل (3, 7) من المثال المحلول؛
4. يتضمن جمل assert للشكل ولإجمالي North (مع مراجعته بالقائمة الأصلية)؛
5. ينتهي بخلية Markdown فيها جملتان لمديرك: ماذا يحتوي الدفتر، والشيء الوحيد الذي يجب أن يفعله قبل أن يثق في إعادة التشغيل (فحص إعادة التشغيل وتشغيل الكل)؛
6. يجتاز Restart Kernel and Run All Cells….

---

14. تحدٍّ قصير — الشكل الخاطئ

نشاط تجاري خيالي: Hilltop Honey (من U03). سجّل زميل مبيعات البرطمانات في كشكين في السوق على مدى 5 أيام. كان يتوقع الشكل (2, 5)، لكن الكود يطبع (5, 2):

import numpy as np

sales = np.array([
    [14, 9],
    [11, 12],
    [16, 10],
    [13, 15],
    [20, 18],
])
print(sales.shape)

1. اقرأ البيانات قبل أن تغيّر أي شيء. ماذا تمثّل صفوف هذه المصفوفة، وماذا تمثّل أعمدتها؟
2. أعِد كتابة المصفوفة بحيث تكون الصفوف = الأكشاك والأعمدة = الأيام، بالشكل (2, 5). واكتب القيم بالترتيب الجديد؛ ولا تستخدم أي دالة لم تتعلمها بعد.
3. أضف جملة assert تفحص الشكل الجديد. وأضف أخرى تفحص أن مجموع كل القيم لم يتغير. واحسب هذا المجموع يدويًا أولًا.
4. اشرح في جملة واحدة لماذا "الكود يعمل" لم تكن تعني "الترتيب صحيح".

سلّم إجابتك قبل الحصول على التعليقات. لا يوجد حل هنا.

---

15. المصادر

مطلوب (حوالي 30 دقيقة، بالتوازي مع التجهيز):

• NumPy: Installing NumPy. https://numpy.org/install/. اقرأ الجزء الخاص بـ pip والنصيحة الخاصة بالبيئات. (فُتحت في 15 أكتوبر 2026.)
• توثيق Python: venv — creation of virtual environments. https://docs.python.org/3/library/venv.html. استخدم جدول التفعيل. (فُتحت في 15 أكتوبر 2026؛ وكانت الصفحة تعرض توثيق 3.15.0.)
• NumPy Quickstart (دليل NumPy v2.5)، الجزءان "The basics" و"Array creation". https://numpy.org/doc/stable/user/quickstart.html. (فُتحت في 15 أكتوبر 2026.)

موصى به:

• Project Jupyter: Installing Jupyter. https://jupyter.org/install
• JupyterLab 4.6.4: Notebooks. https://jupyterlab.readthedocs.io/en/stable/user/notebook.html
• Python Packaging User Guide: Installing Packages. https://packaging.python.org/en/latest/tutorials/installing-packages/

مرجع (ابحث فيه عند الحاجة):

• JupyterLab 4.6.4: Commands list. ابحث عن "Restart Kernel and Run All Cells…". https://jupyterlab.readthedocs.io/en/stable/user/commands_list.html
• Python Developer's Guide: Status of Python versions. https://devguide.python.org/versions/
• Using Python on Windows وUsing Python on macOS (توثيق Python). https://docs.python.org/3/using/windows.html · https://docs.python.org/3/using/mac.html

للتعمّق (اختياري):

• Python for Data Analysis، الطبعة الثالثة، الفصل 4 "NumPy Basics"، القسم 4.1 (الـ ndarray). https://wesmckinney.com/book/numpy-basics.html. ولاحظ أن الكتاب كُتب لإصدار pandas 2.0؛ لكن أساسيات NumPy في القسم 4.1 تطابق هذا الدرس.

---

16. شخصية تتابعها / قدوة

Fernando Pérez (اختاره D.World لدرس L21).

• حقائق موثّقة (روجعت في 15 أكتوبر 2026):
  • صفحة التاريخ الخاصة بـ IPython تذكر أن Fernando Pérez بدأ IPython عام 2001 عندما كان طالب دراسات عليا.
  • صفحة "About" الخاصة بـ Project Jupyter تذكر أن Jupyter "born out of the IPython Project in 2014" (وُلد من مشروع IPython عام 2014).
• لماذا ترشّحه D.World: الدفتر الذي جهّزته اليوم يأتي من هذا المسار من العمل. فقد بدأ كأداة شخص واحد للحوسبة العلمية التفاعلية، وأصبح مشروعًا مفتوحًا يُستخدم في كل مجالات علم البيانات.
• ملاحظة صريحة: نستشهد فقط بالحقيقتين السابقتين. ومناصبه الحالية تتغير مع الوقت؛ راجع صفحات IPython وJupyter الرسمية للتفاصيل.
• جرّب هذا: افتح صفحة تاريخ IPython وابحث عن مشكلة واحدة أُنشئ IPython لحلها. واربطها في جملة واحدة بشيء فعلته في دفترك اليوم.

---

17. المصطلحات

المصطلح	معناه في هذا الدرس
الحزمة (Package)	كود منشور ليثبّته الآخرون باستخدام pip (مثل NumPy)
البيئة الافتراضية (Virtual environment، venv)	مجلد مستقل له Python وحزم خاصة به، ويُفعَّل في كل طرفية
pip	أداة تثبيت الحزم في Python؛ واستخدمها بصيغة python -m pip ...
دفتر Jupyter (Jupyter notebook)	مستند من خلايا كود وخلايا Markdown، يُحفظ بامتداد .ipynb
الخلية (Cell)	كتلة واحدة في الدفتر: كود أو Markdown
النواة (Kernel)	عملية Python التي تعمل خلف الدفتر، وتشغّل الكود وتتذكر المتغيرات
الحالة المخفية (Hidden state)	متغيرات ما زالت النواة تتذكرها رغم أن الخلايا التي أنشأتها غُيّرت أو أُعيد ترتيبها أو حُذفت
الاستيراد / الاسم المختصر (import / alias)	تحميل حزمة (import numpy) / اسمها المختصر (as np)
مصفوفة NumPy (ndarray)	شبكة من القيم تشترك كلها في نوع بيانات واحد
العنصر (Element)	قيمة واحدة داخل المصفوفة
الشكل (Shape)	عدد العناصر في كل بُعد، في صورة صف ثابت، مثل (3, 7) = 3 صفوف و7 أعمدة
ndim	عدد الأبعاد (1 لصف من القيم، و2 للشبكة)
size	العدد الإجمالي للعناصر (حاصل ضرب أرقام الشكل)
dtype	نوع البيانات الواحد لكل العناصر، مثل int64 وfloat64 وbool
الصف الثابت (Tuple)	تسلسل لا يمكن تغييره بين أقواس دائرية؛ و(7,) فيه عنصر واحد
الترقية (Upcasting)	اختيار NumPy نوع بيانات أوسع (مثل float64) حتى تتسع له كل المُدخلات

مراجعة من المرحلة 1: القائمة والفهرسة (L15) · int/float/bool (L13) · استدعاء الدالة ورسالة الخطأ (L16).

---

18. تأمّل

1. أي خطوة من خطوات التجهيز استغرقت أطول وقت، وماذا ستقول لزميل لتجعلها أسرع؟
2. قبل تشغيل المثال المحلول، هل توقعت أن يكون الشكل (3, 7) أم (7, 3)؟ وماذا سيعني كل ترتيب بالنسبة لـ Cedar Lane؟
3. صِف الحالة المخفية بكلماتك. ومتى يمكن أن تضلّل مديرك؟

---

19. الخطوة التالية

أصبحت الآن قادرًا على بناء شبكة ووصفها بدقة. في L22 — الفهرسة والتقطيع والأقنعة المنطقية، ستختار منها: أسبوع نقطة استلام واحدة، ويوم الجمعة لكل النقاط، وكتلة عطلة نهاية الأسبوع، والخلايا التي تقل عن حد معيّن للمبيعات. ومعرفة الشكل هي ما يتيح لك أن تتوقع ما يُرجعه كل اختيار. احتفظ بـ dworld-env وبدفترك. فدرس L22 يستخدم البيئة نفسها، ولا يحتاج إلى تثبيت أي شيء جديد.

---

20. أدلة التقييم

الدليل	ماذا يُظهر	كيف يُقيَّم
دليل التجهيز: دفتر فيه خلية إصدارات ويجتاز Restart Kernel and Run All Cells	بيئة عمل جاهزة للمرحلة 2 (أو بيئة Jupyter مستضافة)	معايير L21-SETUP (إصدارات ضمن النطاق، وإعادة تشغيل نظيفة)
اختبار الفهم (Q1–Q8)	توقّع الشكل ونوع البيانات؛ والتمييز بين تكرار القائمة والضرب عنصرًا بعنصر؛ وفهم الحالة المخفية	7 إجابات صحيحة أو أكثر من 8
التدريب الموجَّه (الأجزاء 1–5)	بناء مصفوفات أحادية وثنائية البعد وفحصها، ومراجعتها باستخدام Python العادي	معايير L21-GP
التدريب المستقل، ومهمة العمل، والتحدي القصير	اختيار شكل وتبريره دون تلميحات؛ وإنشاء دفتر يستطيع الآخرون إعادة تشغيله	معايير L21-IP وL21-WT وL21-MC

أدلة هذا الدرس تُحسب ضمن U06. وتقييم نهاية الوحدة U06 يأتي بعد L24، وليس جزءًا من هذا الدرس.

---

المراجع

1. NumPy Developers. Installing NumPy. https://numpy.org/install/ (فُتحت في 15 أكتوبر 2026). يدعم: تثبيت NumPy باستخدام pip، والتوصية باستخدام بيئة.
2. NumPy Developers. NumPy 2.5.0 Release Notes. https://numpy.org/doc/stable/release/2.5.0-notes.html (فُتحت في 15 أكتوبر 2026). يدعم: "This release supports Python versions 3.12-3.14."
3. NumPy Developers. NumPy quickstart (دليل v2.5). https://numpy.org/doc/stable/user/quickstart.html (فُتحت في 15 أكتوبر 2026). يدعم: خصائص الـ ndarray ‏ndim وshape وsize وdtype؛ وarray وzeros وones وarange.
4. Python Software Foundation. venv — Creation of virtual environments. https://docs.python.org/3/library/venv.html (فُتحت في 15 أكتوبر 2026؛ توثيق 3.15.0). يدعم: python -m venv؛ وأوامر التفعيل لكل واجهة؛ وملاحظة سياسة التنفيذ في PowerShell.
5. Python Software Foundation. Using Python on Windows. https://docs.python.org/3/using/windows.html (فُتحت في 15 أكتوبر 2026؛ توثيق 3.15.0). يدعم: "To obtain Python from the CPython team, use the Python Install Manager"؛ وpy install؛ وpy -V:3.14؛ وهجر برنامج التثبيت الكامل منذ 3.14؛ وإدخال PATH الاختياري.
6. Python Software Foundation. Using Python on macOS. https://docs.python.org/3/using/mac.html (فُتحت في 15 أكتوبر 2026؛ توثيق 3.15.0). يدعم: برنامج التثبيت .pkg من python.org؛ والمجلد /Applications/Python 3.x؛ وInstall Certificates.command؛ وحساب المسؤول؛ وعدم تعديل /usr/bin/python3 الخاص بـ Apple.
7. Python Software Foundation. Status of Python versions (Developer's Guide). https://devguide.python.org/versions/ (فُتحت في 15 أكتوبر 2026، عبر سجل خريطة المرحلة 2). يدعم: صدور 3.15 في 2026-10-09؛ وأن 3.14 في مرحلة إصلاح الأخطاء (bugfix).
8. Python Packaging Authority. Installing Packages. https://packaging.python.org/en/latest/tutorials/installing-packages/ (عبر سجل خريطة المرحلة 2). يدعم: استخدام pip مع البيئات الافتراضية؛ وصيغة python -m pip.
9. Project Jupyter. Installing Jupyter. https://jupyter.org/install (عبر سجل خريطة المرحلة 2). يدعم: pip install jupyterlab، ثم jupyter lab.
10. Project Jupyter. JupyterLab 4.6.4: Notebooks وCommands list. https://jupyterlab.readthedocs.io/en/stable/user/notebook.html · https://jupyterlab.readthedocs.io/en/stable/user/commands_list.html (فُتحتا في 15 أكتوبر 2026). يدعم: إنشاء دفتر عبر + وLauncher مع نواة؛ وتسميات الأوامر "Restart Kernel and Run All Cells…" و"Change Kernel…".
11. IPython. History. https://ipython.readthedocs.io/en/stable/about/history.html · Project Jupyter. About. https://jupyter.org/about (عبر سجل خريطة المرحلة 2، IPY-HIST وJUP-ABOUT). يدعمان الحقائق الخاصة بالقدوة.
12. McKinney, W. Python for Data Analysis، الطبعة الثالثة، الفصل 4. https://wesmckinney.com/book/numpy-basics.html (عبر سجل خريطة المرحلة 2، WM-CH4).
13. D.World Stage 2 Detailed Lesson Map v1.0 (صف L21 ومتطلب التجهيز في U06)، وCurriculum Map v1.0، وCurriculum Bible v1.0. وثائق داخلية.

L22 — Indexing, Slicing & Boolean Masks

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U06 Batch L21–L23 v1 · Content date: 15 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13 and NumPy 2.5) on Linux and on Windows 11 on 4 November 2026 (see the Code Test Report).

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	Indexing, Slicing & Boolean Masks
Stage	Stage 2 — Data Handling
Unit	U06 — NumPy Foundations
Lesson ID	L22
Skills	Indexing; slicing
Difficulty	Beginner–intermediate
Estimated time	About 2 hours (planning estimate, unvalidated). Assumes your L21 environment works.
Assessment level	Practice
Prerequisites	L21: arrays, shape and dtype, and a working environment. U04: zero-based indexing (L15); comparison operators and and/or/not (L14).

Time breakdown (planning estimate):

• Concept and worked example: 40 min.
• Guided practice: 25 min.
• Independent practice: 20 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 10 min (optional).
• Workplace task and mini challenge: 15 min.

What you need: your dworld-env environment and JupyterLab from L21, or your hosted Jupyter environment. You do not need to install anything new.

---

2. Learning Objective

By the end of this lesson, you will be able to select single values, rows, columns and sub-blocks of a 2-D array using integer indexing, slicing and Boolean masks. You will also be able to state correctly whether each result is a view or a copy.

D.World approved objective (verbatim): Select single values, rows, columns and sub-blocks of a 2-D array with integer indexing, slicing and Boolean masks, and state correctly whether each result is a view or a copy.

---

3. Why It Matters

A grid of numbers only helps when you can ask it precise questions:

• "What did Central sell on Friday?"
• "Show me every point's weekend."
• "Which cells were below 10 units?"

Each of these is a selection. Writing the selection wrong gives a wrong answer without any error message. That happens if you are off by one day, take a column instead of a row, or join two conditions incorrectly.

There is a second, quieter risk. Some selections are views onto the original data. If you change a view, the original changes too. In a shared notebook, that can silently alter the figures your manager sees.

Key principle: translate the question into index ranges first, then check the result's shape and one value by eye.

---

4. Concept

All examples use the L21 grid. Rows are pickup points: North, Central, Harbor. Columns are days: Mon … Sun.

import numpy as np

units = np.array([
    [12, 15, 11, 14, 18, 25, 22],   # North
    [20, 18, 22, 19, 31, 30, 28],   # Central
    [ 8,  0,  9, 10, 12, 16, 17],   # Harbor
])
days = np.array(["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"])
print(units.shape, days.shape)

(3, 7) (7,)

days is an array of text, with dtype <U3. You will use it to turn positions into day names.

Part A — Integer indexing: one value, one row, one column

In a 2-D array, a[i, j] means row i, column j. Counting starts at 0, as with lists in L15.

Expression	Means	Result shape
units[1, 4]	row 1 (Central), column 4 (Fri): one value	a single value
units[0]	row 0 (North): the whole week	(7,)
units[:, 4]	every row, column 4: every point's Friday	(3,)

print(units[1, 4])
print(units[0])
print(units[:, 4])
print(units[0].shape, units[:, 4].shape)

31
[12 15 11 14 18 25 22]
[18 31 12]
(7,) (3,)

• The colon : on its own means "all of this axis". So units[:, 4] reads as "all rows, column 4".
• Off-by-one trap. Friday is the 5th day of the week but has index 4. When a question says "day 5", decide whether it means a position or a label, and write that down.
• Negative indices count from the end: units[-1] is the last row (Harbor) and units[:, -1] is the last column (Sunday).

print(units[-1])
print(units[:, -1])

[ 8  0  9 10 12 16 17]
[22 28 17]

Part B — Slicing: blocks of rows and columns

A slice is written start:stop:step:

• It includes start and excludes stop, like range() in L15.
• If you leave out start, it begins at the start of the axis. If you leave out stop, it runs to the end. If you leave out step, the step is 1.
• In 2-D, you write one slice per axis, separated by a comma.

Expression	Means	Shape
units[:, 5:7]	all points, Sat and Sun (columns 5 and 6)	(3, 2)
units[:, -2:]	the same weekend block, counted from the end	(3, 2)
units[1:, :3]	Central and Harbor, Mon–Wed	(2, 3)
units[0, ::2]	North, every second day: Mon, Wed, Fri, Sun	(4,)

weekend = units[:, 5:7]
print(weekend)
print(weekend.shape)
print(units[1:, :3])
print(units[0, ::2])

[[25 22]
 [30 28]
 [16 17]]
(3, 2)
[[20 18 22]
 [ 8  0  9]]
[12 11 18 22]

An index drops a dimension; a slice keeps it. units[0] has shape (7,). units[0:1] selects the same numbers, but as a grid with one row, so its shape is (1, 7). This difference matters in L24.

print(units[0].shape, units[0:1].shape)

(7,) (1, 7)

Slices that run past the end do not raise an error. They just stop at the edge. A single index past the end does raise an error:

print(units[0, 5:100])

[25 22]

units[0, 7]

IndexError: index 7 is out of bounds for axis 1 with size 7

The error message names the axis (1 = columns) and its size (7). Read it the way you learned in L16: last line first.

Part C — Comparisons make Boolean arrays

Comparing an array with a number compares every element and gives an array of True/False with the same shape:

low = units < 10
print(low)
print(low.dtype, low.shape)

[[False False False False False False False]
 [False False False False False False False]
 [ True  True  True False False False False]]
bool (3, 7)

Part D — Boolean masks select values

Putting a Boolean array inside the brackets keeps only the elements where it is True. Used this way, it is called a mask:

print(units[units < 10])
print(units[units < 10].shape)

[8 0 9]
(3,)

• The result is always 1-D. The grid layout is gone, because the selected values do not form a rectangle. You get the values in row order: all of North first, then Central, then Harbor.
• A mask on one row works the same way. Combined with days, it answers "which days?":

print(days[units[2] < 10])

['Mon' 'Tue' 'Wed']

This works because units[2] < 10 has shape (7,), the same as days. The mask and the array you apply it to must match in shape.

A mask that selects nothing is not an error. It returns an empty array:

nothing = units[units > 100]
print(nothing, nothing.shape)

[] (0,)

Check for this case. A report that says "the low days were: " followed by nothing is usually more confusing than one that says "no day was below 100".

Part E — Combining conditions: &, |, ~ and parentheses

For arrays, you cannot use the words and, or and not from L14. Use these symbols instead:

Meaning	For single values (L14)	For arrays (element by element)
both	and	&
either	or	\|
not	not	~

Each condition must be in parentheses.

mid = units[(units >= 10) & (units < 15)]
print(mid)
either = units[(units < 10) | (units > 28)]
print(either)
print(units[2][~(units[2] < 10)])

[12 11 14 10 12]
[31 30  8  0  9]
[10 12 16 17]

Two mistakes that raise the same error. Using and makes Python ask whether the whole array is true, which has no single answer:

(units >= 10) and (units < 15)

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Forgetting the parentheses gives the same error for a less obvious reason: & is applied before >= and <, so Python first calculates 10 & units.

units >= 10 & units < 15

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

If you see this message, check for and/or or missing parentheses first.

Part F — Selecting several rows or columns by position

You can also put a list of positions inside the brackets. This is called advanced or fancy indexing:

print(units[[0, 2]])        # North and Harbor
print(units[:, [0, 6]])     # every point's Monday and Sunday

[[12 15 11 14 18 25 22]
 [ 8  0  9 10 12 16 17]]
[[12 22]
 [20 28]
 [ 8 17]]

Part G — Views and copies

This is the part that causes silent mistakes.

How you select	Example	You get a…	Changing the result changes the original?
Basic slicing (colons and plain integers)	units[:, 5:7], units[0]	view	Yes
Boolean mask	units[units < 10]	copy	No
Advanced indexing (a list of positions)	units[[0, 2]]	copy	No
Explicit copy	units[:, 5:7].copy()	copy	No

A view is a new window onto the same numbers in memory, so nothing is duplicated. NumPy's documentation states that basic slicing always returns views, while advanced indexing (including Boolean masks) always returns copies.

The modification test. This is the reliable way to check, and you will use it throughout U06. To protect the real data, work on a copy of the grid:

grid = units.copy()                 # a separate copy to experiment on

wknd_view = grid[:, 5:7]            # basic slice -> view
wknd_view[0, 0] = 0                 # change the view...
print(grid[0])                      # ...and the original changed too

wknd_copy = units[:, 5:7].copy()    # explicit copy
wknd_copy[0, 0] = 0
print(units[0])                     # the original is unchanged

low_vals = units[units < 10]        # Boolean mask -> copy
low_vals[0] = 99
print(units[2])                     # unchanged

[12 15 11 14 18  0 22]
[12 15 11 14 18 25 22]
[ 8  0  9 10 12 16 17]

Rule of thumb: if you plan to change a selection and the original must stay as it is, call .copy() explicitly.

Selecting vs assigning. low_vals = units[units < 10] selects values, and the result is a copy. Writing a mask on the left of = is different: units[units < 10] = 10 assigns to the original array and changes it. Overwriting data like this is a cleaning decision. U09 teaches when it is justified. Do not do it in U06.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. The manager asks two questions:

1. "How did each point do at the weekend?"
2. "Which of Harbor's days were below 10 units?"

Step 1 — Translate the questions into index ranges before writing any code.

Question	Rows	Columns	Expected shape
Weekend, every point	all → :	Sat, Sun = indices 5 and 6 → 5:7	(3, 2)
Harbor's days below 10	row 2	a mask on that row: units[2] < 10	(number of such days,)

Step 2 — Expected answer first. Read the values off the grid:

• The weekend block is North 25, 22; Central 30, 28; Harbor 16, 17.
• Harbor's week is 8, 0, 9, 10, 12, 16, 17. Below 10 are 8, 0 and 9, on Mon, Tue and Wed. So we expect 3 days.

Step 3 — Code, with checks.

weekend = units[:, 5:7]
assert weekend.shape == (3, 2)
print(weekend)

harbor = units[2]
harbor_low_days = days[harbor < 10]
print(harbor_low_days)
assert harbor_low_days.size == 3
print("checks passed")

[[25 22]
 [30 28]
 [16 17]]
['Mon' 'Tue' 'Wed']
checks passed

Step 4 — Read one value back off the printed grid. weekend[1, 0] should be Central's Saturday. On the grid, that is 30. ✔

Step 5 — View or copy? weekend was made by basic slicing, so it is a view. If you were going to edit it (you are not), you would make a .copy() first.

---

6. Visual Explanation

              Mon Tue Wed Thu Fri Sat Sun
  index  →     0   1   2   3   4   5   6   (−2, −1 count from the end)
  0 North     12  15  11  14  18 [25  22]
  1 Central   20  18  22  19 (31)[30  28]
  2 Harbor    {8}{0} {9} 10  12 [16  17]

  (31)        units[1, 4]       one value            → 31
  [ ... ]     units[:, 5:7]     weekend block, VIEW   → shape (3, 2)
  { ... }     units[units < 10] Boolean mask, COPY    → [8 0 9]  shape (3,)

  slice 5:7  →  includes 5, stops BEFORE 7

Alt text: The 3 by 7 sales grid with the column indices 0 to 6 written above the day names, and a note that −2 and −1 count from the end. Three selections are marked. First, the single cell in row 1, column 4 (Central, Friday) holds 31; it is selected by units[1, 4]. Second, the block of Saturday and Sunday for all three points is selected by units[:, 5:7]. It is a view with shape (3, 2). Third, Harbor's Monday, Tuesday and Wednesday values (8, 0 and 9) are selected by the mask units < 10. That gives a copy, the 1-D array [8 0 9] with shape (3,). A note at the bottom: the slice 5:7 includes 5 and stops before 7.

---

7. Guided Practice

Use your L21 notebook environment. Start with a code cell that creates units and days exactly as in Section 4, or open L22_starter.ipynb from the practice files. For each step, write your expected shape and values in a Markdown cell before you run the code.

1. One point's week. Select Central's full week. What shape do you expect?
2. All points on day 5 of the week. Day 5 is Friday. Which index is that? Select it for every point and check the shape.
3. The weekend block. Select Saturday and Sunday for every point in two different ways: once with positive indices and once with negative indices. Use assert to show that both give identical values. (Hint: (a == b).all() is True when every element matches. .all() comes up properly in L23.)
4. Low-sales cells. Your manager defines "low" as fewer than 12 units. Create the mask. Print the selected values and their shape.
5. Low days for one point. Use days and a mask to list North's low days (fewer than 12 units).
6. Run Restart Kernel and Run All Cells… and confirm that everything still runs.

---

8. Independent Practice

Answer each question with one selection expression. Before running it, write the index ranges you chose and why you chose them. Use the units and days arrays from Section 4.

1. On which days did Harbor sell zero units? (In L23 you will extend this to "on which days did any point sell zero?")
2. What were Central's sales from Tuesday to Thursday, inclusive?
3. What did every point sell on Sunday?
4. On which days did North sell 15 or more units and fewer than 25?

View vs copy test.

5. Make test_grid = units.copy(). Select North's Mon–Wed block from test_grid with a slice. Set the first element of your selection to -1, then print test_grid[0]. What happened, and why?
6. Repeat step 5, but this time prevent the change from reaching test_grid. Show the evidence.

Edge cases.

7. Write a mask that selects nothing from units. Print the result and its shape.
8. Trigger an IndexError on purpose, with an index that does not exist. Copy the last line of the error message and explain which axis it refers to.

No model answer is shown here.

---

9. Knowledge Check — Output Prediction

Use the units and days arrays from Section 4. Write each prediction before running anything.

Q1. What does print(units[2, 1]) print?

• A) 18
• B) 0
• C) 15
• D) IndexError

Q2. What is units[:, 1].shape?

• A) (7,)
• B) (3, 1)
• C) (3,)
• D) (1, 3)

Q3. What does print(units[0, 2:5]) print?

• A) [15 11 14]
• B) [11 14 18]
• C) [11 14 18 25]
• D) [22 19 31]

Q4. What does this print?

g = units.copy()
s = g[0, :3]
s[0] = 100
print(g[0, 0])

• A) 12
• B) 100
• C) 0
• D) An error

Q5. What does this print?

g = units.copy()
m = g[g > 28]
m[0] = 0
print(g[1, 4])

• A) 0
• B) 31
• C) 30
• D) An error

Q6. What happens when you run units[(units > 10) and (units < 20)]?

• A) It returns the values between 10 and 20.
• B) It returns an empty array.
• C) It raises a ValueError, because and cannot combine arrays; use & with parentheses.
• D) It raises an IndexError.

Q7. What does print(days[units[1] >= 28]) print?

• A) ['Fri' 'Sat' 'Sun']
• B) ['Sat' 'Sun']
• C) [31 30 28]
• D) ['Thu' 'Fri' 'Sat']

Q8. Which of these is a view of units?

• A) units[units < 10]
• B) units[[0, 1]]
• C) units[1:, 2:4]
• D) units[:, 2:4].copy()

---

10. AI in Practice (optional, bounded)

Goal: ask an AI assistant to turn a plain-English selection into an expression, then check the index range yourself.

1. Paste only this synthetic context:

"I have a NumPy array units with shape (3, 7). Rows are 3 pickup points and columns are 7 days, Monday to Sunday. Write one NumPy expression that selects the last three days for every point. Explain which index range you used and what shape the result should have."

2. Before you run its answer, write down which three days "the last three days" means (names and indices), and the shape you expect.
3. Run its expression on units. Then follow Section 11.

Boundaries:

• AI is optional.
• Paste only synthetic arrays like this one, never real sales or customer data.
• Do not accept an answer you have not run and checked.

---

11. AI Verification

1. Index range. "Last three days" is Fri, Sat, Sun, which is indices 4, 5, 6: units[:, 4:7] or units[:, -3:]. Common AI slips are -3:-1, which stops before Sunday, and [:, 5:7], which gives only two days. Check the stop value against "excludes stop".
2. Shape. Print the result's shape. You expect (3, 3). If it is (3,) or (3, 2), the selection is wrong even though no error appeared.
3. Read one value off the printed array. Central's Sunday should be 28. Find it in the result.
4. View or copy? If the AI's code changes the result, apply the modification test from Section 4 Part G. Add .copy() where the original must stay unchanged.
5. Edge cases. Try a mask that selects nothing, and an out-of-range index. Does the AI's approach behave sensibly (an empty result; an IndexError)?
6. Explain it back. In one sentence of your own, say why the expression is correct. If you cannot, you have not verified it yet.

---

12. Business Lens

Flag low-sales days for a staffing review. Cedar Lane staffs each pickup point every day. On a day with very low sales, the staff time may be better spent elsewhere. Before anyone changes a rota, the manager wants the evidence:

• Which point, which days, how low? A mask such as days[units[2] < 10] gives the exact days in one line, with no manual scanning.
• A clear threshold. "Low" must be a number the manager agrees on (10? 12?). The mask makes the rule explicit and repeatable. If the threshold changes, the answer updates.
• Weekend vs weekday. A slice such as units[:, 5:7] lets the manager compare weekend demand directly.
• One week is thin evidence. One week of synthetic data cannot justify a staffing change on its own. A real review would look at several weeks and at the reasons for a zero. Was the point closed? Was the data missing? Both questions come back in U09.

---

13. Workplace Task — Staffing Review Shortlist

Fictional business: Cedar Lane. Your manager writes:

"For Friday's staffing meeting, I need, for each pickup point, the days with fewer than 12 units, plus the weekend figures. Keep it short."

In a notebook L22_staffing_shortlist.ipynb:

1. Create units and days, with a Markdown note stating that the data is synthetic and the threshold is 12 units.
2. For each point, print its name and its low days. Use a mask on that point's row together with days.
3. Print the weekend block, with a shape assertion.
4. For one point, cross-check the result by reading the row by eye, and record that you did.
5. End with a three-sentence note to the manager: what you found, the threshold you used, and one reason why a low day might not mean low demand.
6. Run Restart Kernel and Run All Cells….

---

14. Mini Challenge — Two Bugs, No Error Messages

Fictional business: Hilltop Honey. A colleague wants both stalls' sales for days 2 to 4 of a 5-day market. They mean the 2nd, 3rd and 4th days, inclusive. They also want the cells with at least 15 jars. Their code runs without an error, but both answers are wrong:

import numpy as np

jars = np.array([[15, 12, 17,  9, 21],
                 [10, 13, 11, 16, 19]])
middle = jars[:, 2:4]
busy = jars[jars > 15]
print(middle)
print(busy)

1. Before you run the code, write the values you expect for each question by reading the grid.
2. Find the two bugs. One is an off-by-one in the slice and one is in the comparison. Explain each in one sentence.
3. Fix both, and add a shape assertion for middle.
4. Is middle a view or a copy? Prove it with a modification test on a .copy() of jars.

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 25 min):

• NumPy: Indexing on ndarrays. Read the sections "Basic indexing" (single element indexing and slicing) and "Advanced indexing" (Boolean array indexing), plus the opening note about views and copies. https://numpy.org/doc/stable/user/basics.indexing.html (Opened 15 Oct 2026.)

Recommended:

• NumPy Quickstart (NumPy v2.5 manual), sections "Indexing, slicing and iterating" and "Copies and views". https://numpy.org/doc/stable/user/quickstart.html

Go deeper (optional):

• Python for Data Analysis, 3rd ed., Ch. 4 "NumPy Basics", section on basic indexing and slicing and on Boolean indexing. https://wesmckinney.com/book/numpy-basics.html. The book targets pandas 2.0, but its NumPy indexing material matches this lesson.

---

16. Person to Follow / Role Model

Travis Oliphant (mapped by D.World for U06).

• Verified facts (checked 15 Oct 2026): NumPy's "About Us" page lists him as "project founder, 2005-2012". The same page says NumPy was created in 2005, building on the earlier Numeric and Numarray libraries.
• Why D.World recommends him: the array, the indexing rules and the view/copy behaviour you used today are part of the library he founded. NumPy now underpins pandas and much of the Python data stack.
• Honest note: we cite only what the NumPy page states. NumPy today is maintained by a large community, and his current roles change over time.
• Try this: open the NumPy "About Us" page and find how NumPy describes its governance. In one sentence, explain why a library that many projects depend on might need a community rather than one founder.

---

17. Glossary

Term	Meaning in this lesson
Index	A position number; a[i, j] = row i, column j, counting from 0
Negative index	A position counted from the end: -1 is the last
Slice	start:stop:step; includes start, excludes stop
Element-wise comparison	Comparing every element with a value; gives a Boolean array of the same shape
Boolean mask	A True/False array used inside [ ] to keep only the True positions; the result is 1-D
&, \|, ~	Element-wise "and", "or", "not" for arrays; each condition in parentheses
Advanced (fancy) indexing	Selecting with a list of positions (or a Boolean array)
View	A selection that shares memory with the original; changing it changes the original
Copy	A selection with its own memory; changing it leaves the original unchanged
.copy()	Makes an explicit copy
IndexError	Raised when an index is outside an axis; the message names the axis and its size

Review: zero-based indexing (L15) · comparison operators and and/or/not (L14) · shape (L21).

---

18. Reflection

1. Which selection did you get wrong at first: an off-by-one, a row/column swap, or a combined condition? What check would have caught it fastest?
2. Explain view vs copy to a colleague who has never used NumPy, in two sentences and without code.
3. When would a view be the right choice, rather than a copy?

---

19. Next Step

You can now pick out exactly the numbers a question needs. In L23 — Vectorized Operations & Aggregation, you will compute on those selections. You will calculate revenue and gross profit for every cell at once, totals per point and per day, and the best day per point. You will also prove that your NumPy code gives the same answers as a Python loop. The masks from this lesson come back as filters on those calculations.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Guided selection practice (steps 1–6)	Selects rows, columns, blocks and masked values; predicts shapes first	Rubric L22-GP
Knowledge check (Q1–Q8): output prediction	Index and slice ranges; view vs copy; & vs and	7 or more of 8 correct
Independent practice (questions 1–8)	Translates plain-English questions into selections with justified ranges; proves view vs copy	Rubric L22-IP
Workplace task, mini challenge	Applies masks to a business question; finds silent selection bugs	Rubrics L22-WT, L22-MC

This lesson's evidence contributes to U06. The U06 Exit Check comes after L24.

---

References

1. NumPy Developers. Indexing on ndarrays (NumPy user guide). https://numpy.org/doc/stable/user/basics.indexing.html (opened 15 Oct 2026, N-IDX). Supports: basic slicing returns views; advanced indexing (including Boolean arrays) returns copies; slice semantics; negative indices; out-of-bounds indices raise IndexError.
2. NumPy Developers. NumPy quickstart (v2.5 manual). https://numpy.org/doc/stable/user/quickstart.html (R03). Supports: indexing, slicing and iterating; copies and views; .copy().
3. NumPy Developers. About Us. https://numpy.org/about/ (NP-ABOUT, via the Stage 2 map log). Supports the role-model facts.
4. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 4. https://wesmckinney.com/book/numpy-basics.html (WM-CH4).
5. Python Software Foundation. Expressions — Operator precedence (Python Language Reference). https://docs.python.org/3/reference/expressions.html#operator-precedence. Supports: & binds more tightly than comparisons, which is why each condition needs parentheses. (Added for this lesson; see the Source and Assumption Log.)
6. D.World Stage 2 Detailed Lesson Map v1.0 (L22 row). Internal.

L23 — Vectorized Operations & Aggregation

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U06 Batch L21–L23 v1 · Content date: 15 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13 and NumPy 2.5) on Linux and on Windows 11 on 4 November 2026 (see the Code Test Report).

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	Vectorized Operations & Aggregation
Stage	Stage 2 — Data Handling
Unit	U06 — NumPy Foundations
Lesson ID	L23
Skills	Vectorization; aggregation
Difficulty	Beginner–intermediate
Estimated time	About 2 hours (planning estimate, unvalidated)
Assessment level	Application
Prerequisites	L21 (arrays, shape, dtype) and L22 (indexing, slicing, masks). U04: loops and functions (L15, L16) and assert (L16). U05: testing (L19). U02: gross profit and margin (L06).

Time breakdown (planning estimate):

• Concept and worked example: 40 min.
• Guided practice: 20 min.
• Independent practice (loop rewrite with tests): 25 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 10 min (optional).
• Workplace task and mini challenge: 15 min.

What you need: your dworld-env environment from L21, or your hosted Jupyter environment. You do not need to install anything new.

---

2. Learning Objective

By the end of this lesson, you will be able to:

• rewrite an explicit Python loop as an equivalent vectorized NumPy expression;
• prove that the two are equal with tests;
• compute sums, means, minima, maxima and argmax along a chosen axis, and explain the shape of each result.

D.World approved objective (verbatim): Rewrite an explicit Python loop as an equivalent vectorized NumPy expression, prove equality with tests, and compute sums, means, minima, maxima and argmax along a chosen axis, explaining the shape of each result.

---

3. Why It Matters

In U04 you wrote loops to calculate revenue and gross profit one item at a time. Loops are clear, but they get long. Every loop is also a place where an index can slip.

NumPy lets you write the whole calculation once for every element: units * price. This is called vectorized code. It is usually shorter and closer to the formula you would write on paper. For large arrays it is also typically much faster. This lesson does not quote speed figures, because they depend on your computer.

Shorter is not automatically correct. When you replace working code, you must show that the new version gives the same answers. Then aggregation lets you summarise the grid: totals per point, totals per day, and the best day. Here the most common mistake is choosing the wrong axis. It gives a plausible-looking answer to the wrong question.

Key principle: correctness first. Prove the vectorized code equals the loop, and state which axis you are collapsing and why.

---

4. Concept

Start each session with the L22 arrays:

import numpy as np

units = np.array([
    [12, 15, 11, 14, 18, 25, 22],   # North
    [20, 18, 22, 19, 31, 30, 28],   # Central
    [ 8,  0,  9, 10, 12, 16, 17],   # Harbor
])
days = np.array(["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"])
points = np.array(["North", "Central", "Harbor"])

Part A — Element-wise arithmetic on arrays of the same shape

When two arrays have the same shape, + - * / pair up elements position by position. The fictional products and prices are from U02–U04:

Product	Units sold this week	Price (USD)	Unit cost (USD)
Notebook	120	12.00	5.00
Planner	45	20.00	8.00
Pen set	80	8.00	3.00

units_sold = np.array([120, 45, 80])
price      = np.array([12.0, 20.0, 8.0])
unit_cost  = np.array([5.0, 8.0, 3.0])

revenue = units_sold * price
print(revenue)

[1440.  900.  640.]

Element 0 is 120 × 12.00 = 1,440. Element 1 is 45 × 20.00 = 900. Element 2 is 80 × 8.00 = 640. Each position uses only its own product's numbers. No loop is needed. You will calculate gross profit per product yourself in Guided Practice.

An array and a single number also combine element by element. You saw this with * 2 in L21. Every cell of the grid at one price of 12.00 USD:

revenue_grid = units * 12.0
print(revenue_grid[0])

[144. 180. 132. 168. 216. 300. 264.]

Arrays of different shapes follow a rule called broadcasting. That rule is L24. In this lesson, combine only arrays of the same shape, or an array and a single number.

Part B — Universal functions (ufuncs)

NumPy's mathematical functions also work element by element. They are called universal functions, or ufuncs.

ufunc	Does	Example
np.round(a, n)	Round each element to n decimals	np.round(units.mean(axis=1), 1)
np.abs(a)	Distance from zero (removes the sign)	np.abs(units[0] - 15) → distance from a 15-unit target
np.sqrt(a)	Square root of each element	np.sqrt(np.array([16.0, 25.0])) → [4. 5.]

print(np.abs(units[0] - 15))
print(np.sqrt(np.array([16.0, 25.0])))

[ 3  0  4  1  3 10  7]
[4. 5.]

np.round rounds a value exactly halfway to the nearest even number. For example, np.round(2.5) gives 2.0. Python's built-in round does the same. Round only for display; keep unrounded values for calculations (as in L13).

Part C — Loop vs vectorized: prove they are equal

Here is a U04-style loop and its vectorized version:

def revenue_loop(units_sold, price):
    result = []
    for i in range(len(units_sold)):
        result.append(units_sold[i] * price[i])
    return result

def revenue_vec(units_sold, price):
    return units_sold * price

loop_result = revenue_loop([120, 45, 80], [12.0, 20.0, 8.0])
vec_result = revenue_vec(np.array([120, 45, 80]), np.array([12.0, 20.0, 8.0]))
print(loop_result)
print(vec_result)

[1440.0, 900.0, 640.0]
[1440.  900.  640.]

They print differently, because one is a list and one is an array. To prove they are equal, use a test, not your eyes:

assert np.shape(loop_result) == vec_result.shape
assert np.allclose(loop_result, vec_result)
print("equal")

equal

Why np.allclose and not ==? Decimal numbers are stored approximately (L13). Two correct calculations done in a different order can differ in the last digit:

three_items = np.array([2.10, 2.10, 2.10])
print(three_items.sum())
print(three_items.sum() == 6.3)
print(np.isclose(three_items.sum(), 6.3))

6.300000000000001
False
True

• np.isclose(a, b) asks whether two values are equal within a tiny tolerance.
• np.allclose(a, b) asks the same question for every element and returns one True or False.
• Always check the shape as well. np.allclose can stretch a shorter input to match a longer one. For example, np.allclose([5], [5, 5]) is True, although the inputs have different lengths. A shape assertion catches that.

Test more than the "happy" input. A vectorized rewrite must also match the loop:

• on zeros;
• on a single element;
• on an empty input, if your data can ever be empty.

You will write these tests in Section 8.

About speed. Vectorized NumPy code is typically much faster than a Python loop on large arrays, because the looping happens inside NumPy's compiled code. The exact gain depends on your data and your computer, so this lesson does not promise a number. In D.World, correctness is checked first; speed matters only once the answer is proven.

Part D — Aggregation: one number from many

An aggregation reduces many values to fewer. With no axis, it uses the whole array:

Method	Returns
.sum()	Total
.mean()	Average
.min() / .max()	Smallest / largest value
.argmax()	Position of the largest value (the first one, if there is a tie)
.cumsum()	Running total

print(units.sum(), units.max(), units.mean())

357 31 17.0

Part E — The axis: which direction do you collapse?

For a 2-D grid, axis=0 and axis=1 say which dimension is collapsed, that is, added up or reduced away:

Call	Collapses	Gives one result per…	Result shape for (3, 7)	Cedar Lane meaning
units.sum(axis=0)	the rows (axis 0, length 3)	column (day)	(7,)	total units per day, all points together
units.sum(axis=1)	the columns (axis 1, length 7)	row (point)	(3,)	total units per point for the week
units.sum()	everything	—	a single value	total units for the week

Shape rule: the axis you aggregate disappears from the shape. Starting from (3, 7), axis=0 leaves (7,) and axis=1 leaves (3,).

per_point = units.sum(axis=1)
per_day = units.sum(axis=0)
print(per_point, per_point.shape)
print(per_day, per_day.shape)

[117 168  72] (3,)
[40 33 42 43 61 71 67] (7,)

Cross-check: both must add up to the same grand total. 117 + 168 + 72 = 357, and 40 + 33 + 42 + 43 + 61 + 71 + 67 = 357. ✔

The same axis rule applies to mean, min, max, argmax and cumsum:

print(np.round(units.mean(axis=1), 1))     # average per point
print(units.max(axis=1))                   # best day's units per point
best_day_index = units.argmax(axis=1)      # position of the best day per point
print(best_day_index)
print(days[best_day_index])                # positions -> day names (L22)
print(units.cumsum(axis=1)[0])             # North's running weekly total

[16.7 24.  10.3]
[25 31 17]
[5 4 6]
['Sat' 'Fri' 'Sun']
[ 12  27  38  52  70  95 117]

• argmax gives a position, not a value. Use it as an index (here, into days) to make it meaningful.
• cumsum keeps the shape (3, 7), because it produces a running total at every position instead of one total.

Aggregating a mask. In a Boolean array True counts as 1, so .sum() counts matches. .any() asks "is at least one True?" and .all() asks "are all True?". With an axis, you can answer "on which days did any point sell zero?":

zero_cells = units == 0
print(zero_cells.sum())                    # how many zero cells in total
print(days[zero_cells.any(axis=0)])        # days where any point sold zero

1
['Tue']

Part F — Edge cases and a preview of missing values

Input	.sum()	.mean()	.max() / .argmax()
One row, shape (1, 7)	works	works	works
All zeros	0	0.0	0 / 0 (first position)
Empty array, shape (0,)	0.0	nan, with a RuntimeWarning	ValueError

empty = np.array([])
print(empty.sum())

0.0

empty.max()

ValueError: zero-size array to reduction operation maximum which has no identity

An empty selection, such as a mask that matched nothing (L22), can therefore crash a report or give nan. Check .size before you aggregate if the selection could be empty.

nan ("not a number") marks a missing or undefined decimal value. It spreads through calculations: np.array([12.0, np.nan]).sum() is nan. Handling missing values properly is U09. For now, recognise nan and treat it as a warning sign.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. The manager asks: "Total units per pickup point and per day, and each point's best day." Notebooks sell at 12.00 USD.

Step 1 — Expected answer first.

• Per point means one result per row, so collapse the columns: axis=1, shape (3,).
• North's total by hand: 12 + 15 + 11 + 14 + 18 + 25 + 22 = 117.
• Per day means one result per column, so collapse the rows: axis=0, shape (7,).
• Friday by hand: 18 + 31 + 12 = 61.
• North's best day by eye: 25, on Saturday.

Step 2 — A loop version first (U04 style), as the trusted reference.

def totals_per_point_loop(grid_rows):
    totals = []
    for row in grid_rows:
        total = 0
        for value in row:
            total = total + value
        totals.append(total)
    return totals

reference = totals_per_point_loop(units.tolist())
print(reference)

[117, 168, 72]

units.tolist() turns the array back into a plain list of lists, so that the loop uses only Stage 1 Python.

Step 3 — The vectorized version, and the test.

per_point = units.sum(axis=1)
assert per_point.shape == (3,)
assert np.allclose(reference, per_point)
assert per_point[0] == 117                  # hand calculation

per_day = units.sum(axis=0)
assert per_day.shape == (7,)
assert per_day[4] == 61                     # Friday by hand
assert per_point.sum() == per_day.sum()     # both directions give the same total

best = days[units.argmax(axis=1)]
for name, total, day in zip(points, per_point, best):
    print(f"{name}: {total} units, best day {day}, revenue {total * 12.0:.2f} USD")

North: 117 units, best day Sat, revenue 1404.00 USD
Central: 168 units, best day Fri, revenue 2016.00 USD
Harbor: 72 units, best day Sun, revenue 864.00 USD

zip walks through several sequences side by side, and :.2f shows two decimals. Both are display helpers only; the calculations are the vectorized lines above.

Step 4 — Compare with the expectation. North 117 ✔, Friday 61 ✔, North's best day Saturday ✔. Every assertion passed.

---

6. Visual Explanation

                     Mon Tue Wed Thu Fri Sat Sun     sum(axis=1)  → one per ROW
          North   [  12  15  11  14  18  25  22 ]  →   117
          Central [  20  18  22  19  31  30  28 ]  →   168          shape (3,)
          Harbor  [   8   0   9  10  12  16  17 ]  →    72
                      ↓   ↓   ↓   ↓   ↓   ↓   ↓
 sum(axis=0) →       40  33  42  43  61  71  67                     shape (7,)
 one per COLUMN

 (3, 7) ──sum(axis=0)──► (7,)        the aggregated axis disappears
 (3, 7) ──sum(axis=1)──► (3,)
 loop result ≈ vectorized result?   assert np.allclose(...)  + check shapes

Alt text: The 3 by 7 sales grid. To the right of each row, an arrow points to that row's total: North 117, Central 168, Harbor 72. This is sum with axis=1, one result per row, shape (3,). Below each column, a downward arrow points to that day's total: 40, 33, 42, 43, 61, 71 and 67. This is sum with axis=0, one result per column, shape (7,). Underneath, two summary lines say that (3, 7) becomes (7,) with axis=0 and (3,) with axis=1, because the aggregated axis disappears. A final line asks whether the loop result is approximately equal to the vectorized result, checked with assert np.allclose plus a shape check.

---

7. Guided Practice

Create units, days and points as in Section 4, or open L23_starter.ipynb. Write the expected shape, and at least one value by hand, before you run each step.

1. Revenue per product. Using units_sold, price and unit_cost from Part A, calculate revenue and gross profit per product, plus the totals. Check one product by hand.
2. Revenue grid. Make revenue_grid = units * 12.0. - Totals per pickup point: which axis? Assert the shape. - Totals per day: which axis? Assert the shape. - Assert that both directions add up to the same total.
3. Best day per point. Use argmax with the correct axis and days to print each point's best day. Then find each point's worst day. (Hint: there is an argmin.)
4. Average per day. Calculate the average units per day across the three points, rounded to 1 decimal for display only. Which day has the highest average?
5. Restart Kernel and Run All Cells…

---

8. Independent Practice — Loop to Vectorized, with Tests

Part 1 — Rewrite a supplied loop. A colleague wrote this function in U04 style:

def gross_profit_loop(units_sold, prices, unit_costs):
    results = []
    for i in range(len(units_sold)):
        results.append(units_sold[i] * (prices[i] - unit_costs[i]))
    return results

1. Write gross_profit_vec(units_sold, prices, unit_costs). It takes NumPy arrays and returns the same results without a loop.
2. Write tests that compare both functions on three inputs: - the three products from Part A; - a case where all units are zero; - a single-element case of your choice.

For each test, assert that the shapes match, and use np.allclose for the values. 3. Hand-calculate the expected result for your single-element case, and assert it too. 4. Extra edge case: what do both functions return for empty inputs? Does your test still pass? Write one sentence about why this case matters for a weekly report.

Part 2 — Convert your own loop. Choose one loop function you wrote in U04 or U05 that works on a list of numbers, for example a total, a revenue list or a discount calculation.

1. Paste it into your notebook unchanged.
2. Write a vectorized version.
3. Write equality tests: at least three inputs, including a zero case and a single element.
4. In two sentences, explain whether the vectorized version is easier or harder to read, and why.

No model answer is shown here.

---

9. Knowledge Check — Axis and Result Shape

Use units (shape (3, 7): rows = points, columns = days) unless a question says otherwise.

Q1. What is the shape of units.sum(axis=0)?

• A) (3,)
• B) (7,)
• C) (3, 7)
• D) a single value

Q2. You want one total per pickup point. Which call is correct?

• A) units.sum(axis=0)
• B) units.sum(axis=1)
• C) units.sum()
• D) units.cumsum(axis=1)

Q3. What does print(units.argmax(axis=1)) print?

• A) [25 31 17]
• B) [5 4 6]
• C) ['Sat' 'Fri' 'Sun']
• D) 4

Q4. What does this print?

a = np.array([[1, 2, 3],
              [4, 5, 6]])
print(a.sum(axis=0), a.sum(axis=1))

• A) [5 7 9] [ 6 15]
• B) [ 6 15] [5 7 9]
• C) 21 21
• D) [5 7 9] 21

Q5. What is the shape of units.cumsum(axis=1)?

• A) (3,)
• B) (7,)
• C) (3, 7)
• D) a single value

Q6. A loop returns [0.30000000000000004] and your vectorized code returns array([0.3]). Which test is the best check that they agree?

• A) assert loop == vec
• B) assert np.allclose(loop, vec), plus a shape check
• C) assert str(loop) == str(vec)
• D) No test; they look the same

Q7. What does this print?

b = np.array([4, 9, 9, 2])
print(b.argmax())

• A) 9
• B) 1
• C) 2
• D) [1 2]

Q8. Which statement is true?

• A) Vectorized code is always correct if it runs without an error.
• B) np.allclose alone proves two results are identical in shape and values.
• C) The axis you aggregate over disappears from the result's shape.
• D) .mean() of an empty array returns 0.

---

10. AI in Practice (optional, bounded)

Goal: ask an AI assistant to vectorize a loop, then test its answer against the loop on your own arrays.

1. Paste only this synthetic code:

"Rewrite this Python function using NumPy so that it has no explicit loop. Inputs are NumPy arrays of the same length. Keep the function name and arguments. Do not add other libraries. python def discounted_revenue_loop(units_sold, prices, discount_rates): out = [] for i in range(len(units_sold)): out.append(units_sold[i] * prices[i] * (1 - discount_rates[i])) return out"

2. Do not trust the answer yet. Write your own test arrays first. Include: - a normal case; - a zero-discount case; - a 100% discount case (1.0); - a single element.
3. Run the loop and the AI's version on every test array, and compare them with Section 11.
4. Extension: ask the AI for the total discounted revenue per pickup point from a (3, 7) grid of revenue values. Check its axis choice.

Boundaries:

• AI is optional.
• Use only synthetic arrays.
• Do not paste real business data.
• Never accept vectorized code you have not compared with a trusted reference.

---

11. AI Verification

1. Run both on the same inputs: assert np.allclose(loop_result, ai_result) for every test array.
2. Check shapes: assert np.shape(loop_result) == ai_result.shape. This catches answers that allclose would wrongly accept because it stretched an input.
3. Hand-calculate one value. For example, 10 units × 4.00 USD × (1 − 0.25) = 30.00 USD.
4. Edge cases: all-zero units, a discount of 0.0 and of 1.0, a single element, and an empty input.
5. Axis choice: for "per pickup point" on a (3, 7) grid, the result must have shape (3,) (that is, axis=1). If the AI used axis=0, the result has shape (7,): totals per day, which answer a different question.
6. Scope: reject code that adds packages, changes the function's inputs, or rounds values inside the calculation.

---

12. Business Lens

The manager's weekly summary: revenue and gross profit per pickup point. Notebooks sell at 12.00 USD and cost 5.00 USD each. These are the U02 L06 definitions, not retaught here: revenue = price × units; gross profit = revenue − cost of goods sold; gross margin = gross profit ÷ revenue.

weekly_units = units.sum(axis=1)               # one total per point
weekly_revenue = weekly_units * 12.0
weekly_gross_profit = weekly_units * (12.0 - 5.0)
print(weekly_revenue)
print(weekly_gross_profit)
print(np.round(weekly_gross_profit / weekly_revenue * 100, 1))

[1404. 2016.  864.]
[ 819. 1176.  504.]
[58.3 58.3 58.3]

• The axis is the business question. axis=1 answers "how did each point do?". axis=0 answers "how did each day go across the business?". Both are useful, for different decisions.
• The same margin everywhere is a clue, not a finding. Every point here sells one product at one price and one cost, so the margins must be equal. Differences between points will appear only when the product mix or the prices differ. Do not present an identical margin as a sign of "consistent performance".
• Tests make the summary trustworthy. If the manager asks "are you sure?", your answer is the assertions: the totals reconcile in both directions, and they match the loop and a hand calculation.

---

13. Workplace Task — Weekly Pickup-Point Summary

Fictional business: Cedar Lane. Your manager writes:

"Please send me, for each pickup point: total units, revenue, gross profit, average units per day and best day. Also the busiest day for the business overall. Notebook price 12.00 USD, unit cost 5.00 USD."

In L23_weekly_summary.ipynb:

1. Calculate every figure with vectorized code and the correct axis. Write a comment on each aggregation that says which axis you collapsed and why.
2. Add assertions: - the shape of each result; - per-point totals sum to the same total as per-day totals; - one total checked against a hand calculation; - one total checked against a loop.
3. Print a short, labelled summary. Use two decimals for money and one decimal for averages, for display only.
4. Add a Markdown note of three sentences to the manager: the main result, the busiest day overall, and one limitation of one week of data.
5. Run Restart Kernel and Run All Cells….

---

14. Mini Challenge — The Plausible Wrong Axis

Fictional business: Hilltop Honey. A colleague wants the average jars sold per stall (2 stalls × 5 days) and gets five numbers instead of two. The code runs without an error:

import numpy as np

jars = np.array([[15, 12, 17,  9, 21],
                 [10, 13, 11, 16, 19]])
avg_per_stall = jars.mean(axis=0)
print(avg_per_stall)

1. Before you run it, predict the shape of the colleague's result. Then predict the shape of the result that the question actually needs.
2. Fix the code, add a shape assertion, and check one stall's average by hand.
3. What business question does the colleague's original output answer? Write it in one sentence.
4. Write one test that would have caught this bug automatically.

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 25 min):

• NumPy Quickstart (NumPy v2.5 manual), sections "Basic operations" (element-wise arithmetic, the axis parameter) and "Universal functions". https://numpy.org/doc/stable/user/quickstart.html (Opened 15 Oct 2026.)

Go deeper (optional):

• Python for Data Analysis, 3rd ed., Ch. 4 "NumPy Basics": section 4.3 (universal functions) and section 4.4 (array-oriented programming, including mathematical and statistical methods). https://wesmckinney.com/book/numpy-basics.html. Written for pandas 2.0; the NumPy material matches this lesson.

Reference (look things up when needed):

• NumPy API pages for numpy.allclose and numpy.isclose, at numpy.org/doc/stable/reference. (Added for this lesson; see the Source and Assumption Log.)

---

16. Person to Follow / Role Model

Travis Oliphant (mapped by D.World for U06).

• Verified facts (checked 15 Oct 2026): NumPy's "About Us" page lists him as "project founder, 2005-2012", and says NumPy was created in 2005, building on the Numeric and Numarray libraries.
• Why D.World recommends him: vectorized, array-oriented computing, which is what you practised today, is the core idea of the library he founded.
• Honest note: this lesson cites only the NumPy page. NumPy is now developed by a broad community, and his current roles change over time.
• Try this: on the NumPy "About Us" page, find one way people can contribute to NumPy that is not writing code. Note it in one sentence.

---

17. Glossary

Term	Meaning in this lesson
Vectorization	Writing a calculation once for whole arrays instead of looping over elements
Element-wise	Applied position by position, to arrays of the same shape (or an array and one number)
Universal function (ufunc)	A NumPy function that works element-wise, e.g. np.round, np.abs, np.sqrt
Aggregation	Reducing many values to fewer: sum, mean, min, max
Axis	The dimension an aggregation collapses: axis=0 = down the rows (one result per column); axis=1 = across the columns (one result per row)
argmax	The position of the largest value (the first one, if tied); argmin for the smallest
Cumulative sum	A running total at each position (cumsum); keeps the shape
Floating-point tolerance	Accepting tiny rounding differences when comparing decimals: np.isclose, np.allclose
nan	"Not a number": marks a missing or undefined decimal; spreads through calculations (U09)

Review: loops and functions (L15, L16) · assert (L16) · gross profit and margin (L06) · testing (L19).

---

18. Reflection

1. Which did you find easier to check: the loop or the vectorized version? What does your answer suggest about when to keep a loop as a reference?
2. Describe axis=0 and axis=1 using Cedar Lane's grid, without the words "row" or "column".
3. Which edge case surprised you most (zeros, a single element, empty), and why?

---

19. Next Step

So far, every calculation combined arrays of the same shape, or an array and one number. Cedar Lane's real questions are messier. Each pickup point has its own cost rate, with shape (3,), which must be applied to the (3, 7) grid. In L24 — Broadcasting, Reshaping & Shape Errors, you will learn NumPy's rule for combining different shapes, read the error that appears when shapes do not fit, and catch the more dangerous case where they fit in the wrong way. The U06 Exit Check comes after L24.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Loop-to-vectorized rewrite with passing tests (Independent Practice, Parts 1–2)	Rewrites a loop; proves equality with shape and allclose tests on normal, zero and single-element inputs	Rubric L23-IP
Knowledge check (Q1–Q8): axis and result shape	Chooses the axis for a question; predicts result shapes; picks a sound equality test	7 or more of 8 correct
Guided practice	Element-wise revenue and gross profit; totals per point and per day; argmax	Rubric L23-GP
Workplace task, mini challenge	Builds a tested weekly summary; diagnoses a wrong-axis bug	Rubrics L23-WT, L23-MC

This lesson's evidence contributes to U06. The U06 Exit Check comes after L24.

---

References

1. NumPy Developers. NumPy quickstart (v2.5 manual). https://numpy.org/doc/stable/user/quickstart.html (opened 15 Oct 2026, R03). Supports: element-wise arithmetic; sum, min, max, cumsum with the axis parameter; universal functions.
2. NumPy Developers. numpy.allclose and numpy.isclose (NumPy API reference). https://numpy.org/doc/stable/reference/generated/numpy.allclose.html · https://numpy.org/doc/stable/reference/generated/numpy.isclose.html. Supports: tolerance-based comparison; inputs are broadcast. (Added; see the Source and Assumption Log.)
3. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 4, sections 4.3–4.4. https://wesmckinney.com/book/numpy-basics.html (WM-CH4).
4. NumPy Developers. About Us. https://numpy.org/about/ (NP-ABOUT). Supports the role-model facts.
5. Python Software Foundation. The Python Tutorial, ch. 15, "Floating-Point Arithmetic: Issues and Limitations." https://docs.python.org/3/tutorial/floatingpoint.html. Supports: decimal values are stored approximately. (Already cited in L13.)
6. D.World Stage 2 Detailed Lesson Map v1.0 (L23 row); U02 L06 metric definitions. Internal.

L24 — Broadcasting, Reshaping & Shape Errors

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U06 L24 + Exit Check v1 · Content date: 15 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13 and NumPy 2.5) on Linux and on Windows 11 on 4 November 2026 (see the Code Test Report).

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	Broadcasting, Reshaping & Shape Errors
Stage	Stage 2 — Data Handling
Unit	U06 — NumPy Foundations
Lesson ID	L24 (last lesson of U06)
Skills	Broadcasting; shapes
Difficulty	Beginner–intermediate
Estimated time	About 2 hours (planning estimate, unvalidated)
Assessment level	Understanding → Application
Prerequisites	L21 (shape, tuples), L22 (indexing and slicing, [:, j]), L23 (element-wise arithmetic, axis). From U04: ValueError and reading tracebacks (L16).

Time breakdown (planning estimate):

• Concept and worked example: 45 min.
• Guided practice: 20 min.
• Independent practice: 20 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 10 min (optional).
• Workplace task and mini challenge: 15 min.

What you need: your dworld-env environment from L21, or your hosted Jupyter environment. You do not need to install anything new.

---

2. Learning Objective

By the end of this lesson, you will be able to:

• predict, using the documented broadcasting rule, whether two shapes are compatible and what shape the result has;
• fix a shape-mismatch error by reshaping or adding an axis, so that values pair as intended.

D.World approved objective (verbatim): Predict, using the documented broadcasting rule, whether two shapes are compatible and what shape the result has, and fix a shape-mismatch error by reshaping or adding an axis so that values pair as intended.

---

3. Why It Matters

In L23 every calculation used arrays of the same shape, or an array and one number. Cedar Lane's real questions mix shapes:

• Each pickup point has its own unit cost. That is three numbers, shape (3,), which must apply to a (3, 7) grid.
• Saturday and Sunday have a promotion. That is seven multipliers, shape (7,), one per day.

NumPy has a rule, broadcasting, for combining arrays of different shapes. When the shapes do not fit the rule, you get an error. That is annoying but safe: you know something is wrong.

The dangerous case is when the shapes do fit, but the values pair up the wrong way. Each cost gets multiplied by the wrong pickup point's sales, or a list of three totals becomes a (3, 3) table. There is no error, and the numbers look plausible.

Key principle: making the error go away is not the goal. The goal is that every value pairs with the right partner. Print the shapes before and after, and check one cell by hand.

---

4. Concept

Start with the U06 arrays:

import numpy as np

units = np.array([
    [12, 15, 11, 14, 18, 25, 22],   # North
    [20, 18, 22, 19, 31, 30, 28],   # Central
    [ 8,  0,  9, 10, 12, 16, 17],   # Harbor
])
days = np.array(["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"])
points = np.array(["North", "Central", "Harbor"])
print(units.shape)

(3, 7)

Part A — Scalar broadcasting (you already know it)

In L21 and L23, units * 12.0 multiplied every cell by 12.0. NumPy's documentation calls this the simplest case of broadcasting: the single number is "stretched" to the array's shape. No copy of it is actually made in memory; NumPy just reuses it.

print((units * 12.0).shape)

(3, 7)

Part B — The broadcasting rule

NumPy compares the two shapes from the right, one dimension at a time. Two dimensions are compatible when:

1. they are equal, or
2. one of them is 1.

If one shape has fewer dimensions, the missing dimensions on the left count as 1. The result takes the larger size in each position. If any pair of dimensions is incompatible, NumPy raises a ValueError.

Worked by hand: (3, 7) with (7,)

  units      3   7
  promo          7      ← missing on the left counts as 1
             ─────
  compare:   3 vs 1 → ok (one is 1)     7 vs 7 → ok (equal)
  result     3   7

(3, 7) with (3,)

  units      3   7
  cost           3
             ─────
  compare:   7 vs 3 → NOT compatible (not equal, neither is 1)  → ValueError

(3, 7) with (3, 1)

  units      3   7
  cost_col   3   1
             ─────
  compare:   3 vs 3 → ok       7 vs 1 → ok
  result     3   7

Shape A	Shape B	Compatible?	Result
(3, 7)	a single number	yes	(3, 7)
(3, 7)	(7,)	yes	(3, 7)
(3, 7)	(3,)	no	ValueError
(3, 7)	(3, 1)	yes	(3, 7)
(3,)	(3, 1)	yes	(3, 3), often a silent mistake (Part F)

Part C — Row vectors and column vectors

A 1-D array such as cost with shape (3,) has no "direction". When it meets a 2-D array, the rule lines it up with the last axis, the columns. To make the direction explicit, give it a second axis:

Name	Shape	Pairs with… in a (3, 7) grid
Row vector	(1, 7), or a plain (7,)	one value per column (per day)
Column vector	(3, 1)	one value per row (per pickup point)

promo = np.array([1.0, 1.0, 1.0, 1.0, 1.0, 0.9, 0.9])   # 10% off at the weekend
cost = np.array([5.0, 4.8, 5.4])                        # USD per notebook, per point
print(promo.shape, cost.shape)

(7,) (3,)

promo has one value per day, so as a row vector it already lines up with the columns. cost has one value per pickup point, so it must become a column vector.

Part D — Three ways to add an axis or change a shape

Tool	Example	Result shape	Use it when…
[:, None]	cost[:, None]	(3, 1)	you want a column vector from a 1-D array
np.newaxis	cost[:, np.newaxis]	(3, 1)	the same thing; np.newaxis is just a readable name for None
[None, :]	promo[None, :]	(1, 7)	you want an explicit row vector
.reshape(r, c)	cost.reshape(3, 1)	(3, 1)	you know the target shape
.reshape(-1, 1)	cost.reshape(-1, 1)	(3, 1)	"as many rows as needed, 1 column"
.reshape(-1)	units.reshape(-1)	(21,)	flatten to 1-D

print(cost[:, None].shape, cost[:, np.newaxis].shape, cost.reshape(3, 1).shape, cost.reshape(-1, 1).shape)
print(cost[:, None])

(3, 1) (3, 1) (3, 1) (3, 1)
[[5. ]
 [4.8]
 [5.4]]

reshape keeps the same values in the same order. It only changes how they are arranged:

six = np.arange(6)
print(six.reshape(2, 3))
print(six.reshape(3, -1).shape)

[[0 1 2]
 [3 4 5]]
(3, 2)

• -1 means "work this one out for me". NumPy calculates it from the total size. Only one dimension can be -1.
• The total size must not change. Six values cannot become 4 rows:

six.reshape(4, -1)

ValueError: cannot reshape array of size 6 into shape (4,newaxis)

Part E — Transpose: .T

.T swaps rows and columns. The (3, 7) grid becomes (7, 3): one row per day and one column per point.

print(units.T.shape)
print(units.T[0])          # Monday, for each point

(7, 3)
[12 20  8]

.T is useful when a colleague's data arrives the other way round (L21's mini challenge). Transposing also changes what each axis means, so restate in words what axis=0 and axis=1 mean after every transpose.

Part F — Reading shape errors, and the silent mistakes

The loud error. Applying the per-point cost directly:

units * cost

ValueError: operands could not be broadcast together with shapes (3,7) (3,)

Read the message from the right, as the rule does:

• The two shapes are listed: (3,7) and (3,).
• Their last dimensions are 7 and 3. They are not equal and neither is 1, so they are incompatible.
• Ask what the 3 means: one cost per pickup point, so one per row. The fix is a column vector, cost[:, None].

The silent mistake, case 1: (3,) with (3, 1) gives (3, 3). The manager has a weekly unit target per point, stored as a column:

per_point = units.sum(axis=1)                 # shape (3,)
targets = np.array([[120], [160], [80]])      # shape (3, 1)
gap = per_point - targets
print(gap.shape)
print(gap)

(3, 3)
[[ -3  48 -48]
 [-43   8 -88]
 [ 37  88  -8]]

You expected three gaps, one per point, and got nine numbers with no error. Each of the three totals was compared with each of the three targets. Only the diagonal (−3, 8, −8) is meaningful. The fix is to make both arrays the same shape:

gap = per_point - targets.reshape(-1)
assert gap.shape == (3,)
print(gap)

[-3  8 -8]

The silent mistake, case 2: a square grid hides the wrong direction. With 3 points and 3 days, cost of shape (3,) is compatible, but it lines up with the days:

small = units[:, :3]                          # 3 points x 3 days (Mon-Wed)
wrong = small * cost                          # pairs cost with DAYS - no error
right = small * cost[:, None]                 # pairs cost with POINTS
print(wrong[1, 0], right[1, 0])               # Central, Monday

100.0 96.0

Check it by hand. Central sold 20 on Monday at Central's cost of 4.80, so the correct value is 20 × 4.80 = 96.00. The wrong version used North's cost of 5.00. Only the hand check reveals this. Both results have shape (3, 3).

Habit for every broadcast:

1. Print the shapes before the operation.
2. Assert the expected result shape.
3. Check one cell by hand, choosing a cell that is not in the first row or column.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. The manager wants this week's gross profit per pickup point. Price is 12.00 USD per notebook. Unit cost differs by point: North 5.00, Central 4.80, Harbor 5.40.

Step 1 — Expected answer first. For North, by hand:

• Revenue = 117 units × 12.00 = 1,404.00.
• Cost = 117 × 5.00 = 585.00.
• Gross profit = 819.00.

The result should have shape (3,), one value per point.

Step 2 — Shapes before.

print(units.shape, cost.shape)

(3, 7) (3,)

The cost is one value per row, so it needs a column vector, (3, 1).

Step 3 — Calculate, with assertions.

revenue_grid = units * 12.0
cost_grid = units * cost[:, None]
assert cost_grid.shape == (3, 7)

gross_profit = (revenue_grid - cost_grid).sum(axis=1)
assert gross_profit.shape == (3,)
assert np.isclose(gross_profit[0], 819.0)        # North by hand

print(np.round(gross_profit, 2))
print(cost_grid[1, 4])                           # Central, Friday

[ 819.  1209.6  475.2]
148.79999999999998

Step 4 — Check one inner cell by hand. Choose a cell that is not in the first row, so that a wrong pairing would show. Central sold 31 on Friday at 4.80 USD, and 31 × 4.80 = 31 × 4 + 31 × 0.8 = 124 + 24.8 = 148.80. The printed cost_grid[1, 4] is 148.79999999999998. That is 148.80 with a tiny floating-point difference, as in L23, so compare with a tolerance rather than ==:

assert np.isclose(cost_grid[1, 4], 31 * 4.80)
print("Central Friday matches")

Central Friday matches

✔

If the cost had been paired the wrong way, Central's Friday would have used a different point's cost, and this check would fail. (On a (3, 7) grid the wrong pairing raises an error anyway. On a square grid it would not; see Part F.)

Step 5 — Compare with the expectation.

• North's gross profit is 819.00, matching the hand value. ✔
• The result has shape (3,). ✔
• One inner cell matches its hand calculation. ✔

When a printed value and your hand check disagree, stop. Rerun that single cell, print the row it came from, and find the cause. Never adjust the hand calculation to match the screen.

---

6. Visual Explanation

  units (3, 7)                     cost[:, None] (3, 1)      cost_grid (3, 7)
  ┌──────────────────────┐         ┌─────┐                   each ROW × its own cost
  │ 12 15 11 14 18 25 22 │    ×    │ 5.0 │ ─ stretched →     North   × 5.0
  │ 20 18 22 19 31 30 28 │         │ 4.8 │   across 7 days   Central × 4.8
  │  8  0  9 10 12 16 17 │         │ 5.4 │                   Harbor  × 5.4
  └──────────────────────┘         └─────┘

  promo (7,) = [1 1 1 1 1 .9 .9]   ─ stretched down 3 rows →  each COLUMN × its factor

  Rule (compare from the RIGHT):   (3, 7)        (3, 7)        (3,)
                                   (   7)  ✔     (   3)  ✘     (3, 1)  ✔ but → (3, 3) ⚠
                                   equal or 1    7 ≠ 3         silent pairing mistake

Alt text: On the left is the 3 by 7 units grid. It is multiplied by the cost column vector of shape (3, 1), holding 5.0, 4.8 and 5.4, which is stretched across all 7 days. The result is a 3 by 7 cost grid in which each row is multiplied by its own pickup point's cost. Below, the promo row vector of shape (7,) is stretched down the 3 rows, so each column (day) is multiplied by its own factor. At the bottom, three shape comparisons are made from the right. (3, 7) with (7,) is compatible, because the last dimensions are equal. (3, 7) with (3,) is incompatible, because 7 is not equal to 3. (3,) with (3, 1) is compatible but produces (3, 3), which is marked as a silent pairing mistake.

---

7. Guided Practice

Create units, days, points, cost and promo as in Section 4, or open L24_starter.ipynb. Write the shapes you expect before each step.

1. Observe the error. Run units * cost. Copy the last line of the error. Explain, using the rule, which two dimensions clash.
2. Fix with reshape. Make the cost a column vector with cost.reshape(3, 1). Multiply, then assert the result shape is (3, 7).
3. Fix with a new axis. Do the same with cost[:, None]. Assert that both fixes give identical results.
4. Check one cell by hand. Choose Harbor's Saturday. Calculate it yourself, then compare it with your result.
5. Per-day promotion. Calculate the weekend-promotion revenue grid units * 12.0 * promo. Predict its shape before running. Explain why promo needs no reshape. Then check Central's Sunday by hand: 28 × 12.00 × 0.9.
6. Both at once. Calculate this week's gross profit per point with the promotion and the per-point cost: - revenue = units × 12.00 × promo; - cost = units × cost per point; - gross profit = revenue − cost, summed per point.

Assert the final shape. 7. Run Restart Kernel and Run All Cells….

---

8. Independent Practice

Part 1 — Classify five shape pairs. For each pair, write first, by hand using the rule, whether it is compatible and, if so, the result shape. Then verify each one by running (np.zeros(a) + np.zeros(b)).shape inside a try/except ValueError (L16).

#	Shape A	Shape B
1	(4, 14)	(14,)
2	(4, 14)	(4,)
3	(4, 1)	(1, 14)
4	(2, 3)	(3, 2)
5	(5,)	(5, 1)

For any pair that is compatible but probably not what a business user intended, add one sentence saying why.

Part 2 — Repair a broken snippet. A colleague wants each pickup point's weekly total as a share of its own target (targets in units). Their code fails:

weekly = units.sum(axis=1)              # one total per point
targets = np.array([120, 160, 80])
share = units / targets                 # intended: weekly total / target, per point

1. Run it and record the error.
2. Two different "fixes" make the error disappear: - (a) units / targets[:, None]; - (b) weekly / targets.

Run both. Print each result's shape. 3. Decide which one answers the colleague's question. Justify it in two or three sentences, using the shapes and the business meaning. 4. Add assertions for the shape and for one value checked by hand.

No model answer is shown here.

---

9. Knowledge Check — Shape Prediction

Write each prediction before running anything. You may use np.zeros(shape) to test after predicting.

Q1. What is the shape of np.zeros((3, 7)) + np.zeros(7)?

• A) (3, 7)
• B) (7, 3)
• C) (7,)
• D) ValueError

Q2. What happens with np.zeros((3, 7)) * np.zeros(3)?

• A) The result has shape (3, 7)
• B) The result has shape (3, 3)
• C) ValueError
• D) The result has shape (3,)

Q3. cost has shape (3,). Which expression gives shape (3, 1)?

• A) cost.T
• B) cost[None, :]
• C) cost[:, None]
• D) cost.reshape(1, 3)

Q4. What is the shape of np.zeros(3) - np.zeros((3, 1))?

• A) (3,)
• B) (3, 1)
• C) (3, 3)
• D) ValueError

Q5. What is the shape of np.arange(12).reshape(-1, 4)?

• A) (4, 3)
• B) (3, 4)
• C) (12, 4)
• D) ValueError

Q6. What does print(units.T.shape) print?

• A) (3, 7)
• B) (7, 3)
• C) (21,)
• D) (7,)

Q7. A 3 × 3 grid (3 points × 3 days) is multiplied by cost of shape (3,). There is no error. What actually happened?

• A) Each point's row was multiplied by its own cost
• B) Each day's column was multiplied by one of the costs, so the costs were paired with days
• C) Nothing; NumPy refuses square grids
• D) The costs were added, not multiplied

Q8. Which is the best evidence that a broadcast fix is correct?

• A) The error message disappeared
• B) The result has the expected shape and one hand-checked inner cell matches
• C) The result has the expected shape
• D) An AI said the fix was correct

---

10. AI in Practice (optional, bounded)

Goal: ask an AI assistant to explain a shape error, then verify that its fix gives the intended shape and the intended pairing.

1. Paste only this synthetic code and error:

"I am learning NumPy. units has shape (3, 7): 3 pickup points (rows) by 7 days (columns). cost has shape (3,): one unit cost per pickup point. This line fails: cost_grid = units * cost Error: ValueError: operands could not be broadcast together with shapes (3,7) (3,) Explain the error using NumPy's broadcasting rule and suggest a fix. Say which dimension each cost should pair with."

2. Before running its fix, write the result shape you expect. Also write the value you expect for Central's Monday: 20 × 4.80.
3. Run the fix, then follow Section 11.

Boundaries:

• AI is optional.
• Use only synthetic arrays like these.
• Do not paste real business data.
• Do not accept a fix only because the error disappeared.

---

11. AI Verification

1. Same error? The explanation must refer to the last dimensions 7 and 3, and say they are incompatible because they are not equal and neither is 1.
2. Shape: assert cost_grid.shape == (3, 7).
3. Pairing: check an inner cell by hand. Central's Monday must be 20 × 4.80 = 96.00, not 20 × 5.00. Then check one Harbor cell.
4. Reject "fixes" that only silence the error. Examples: - units[:, :3] * cost (it throws away four days of data); - transposing without transposing back (the axis meanings change); - np.resize, or repeating the costs to length 7 (the values pair with the wrong days).
5. Explain it back: "the cost must be a column vector (3, 1) because there is one cost per row." If you cannot say this, you have not verified it.

---

12. Business Lens

Point-specific cost rates and day-specific promotions in one step. Real pricing is rarely one number. Each pickup point may have different delivery or handling costs. Promotions run on particular days. Broadcasting lets one line apply the right number to the right cell:

• Per-point values (one per row) become a column vector, (3, 1).
• Per-day values (one per column) become a row vector, (7,) or (1, 7).
• The manager's question decides the direction. "Cost depends on the point" means pair with rows. "The discount depends on the day" means pair with columns.

The risk is a figure that is plausible but wrong: a cost applied to the wrong point, or a gap table with nine numbers instead of three. A finance colleague who finds such a mistake will doubt every other number in the report. Shape assertions and one hand-checked inner cell are the cheapest protection you have.

---

13. Workplace Task — Promotion Impact Check

Fictional business: Cedar Lane. Your manager writes:

"We ran 10% off at the weekend. Using this week's units, what was gross profit per point with the promotion, and how much lower is it than the same units at full price? Use each point's own unit cost."

In L24_promotion_check.ipynb:

1. Create units, cost (per point) and promo (per day), with a Markdown note on what each shape means.
2. Calculate gross profit per point at full price and with the promotion. Use broadcasting, printing the shapes before each operation.
3. Calculate the difference per point.
4. Add assertions: - the result shapes; - one inner cell checked by hand; - one per-point total cross-checked with a plain-Python loop.
5. Write three sentences to the manager: - the result; - which point lost the most gross profit in USD; - one caution. For example: this assumes the same units would have sold without the promotion, which is not known.
6. Run Restart Kernel and Run All Cells….

---

14. Mini Challenge — No Error, Wrong Answer

Fictional business: Hilltop Honey. There are two stalls and two market days. Each stall has its own jar cost: stall 1 pays 4.00 USD per jar and stall 2 pays 3.50 USD. A colleague's code runs without an error:

import numpy as np

jars = np.array([[15, 12],      # stall 1, day 1 and day 2
                 [10, 13]])     # stall 2, day 1 and day 2
jar_cost = np.array([4.00, 3.50])   # stall 1, stall 2
cost_grid = jars * jar_cost
print(cost_grid)

1. Before running it, calculate by hand what stall 2's day 1 cost should be.
2. Run the code. Which cells are wrong, and why did NumPy not raise an error?
3. Fix it so that each stall's cost pairs with its own row. Add a shape assertion and a hand-checked inner cell.
4. In one sentence, explain why square grids are especially risky for broadcasting.

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 25 min):

• NumPy: Broadcasting. Read "General broadcasting rules" and the examples with diagrams. https://numpy.org/doc/stable/user/basics.broadcasting.html (Opened 15 Oct 2026.)

Recommended:

• NumPy Quickstart (NumPy v2.5 manual), sections "Shape manipulation" (reshape, .T) and "Broadcasting rules". https://numpy.org/doc/stable/user/quickstart.html

Reference (look things up when needed):

• NumPy API: numpy.reshape, and numpy.newaxis in the indexing guide (https://numpy.org/doc/stable/user/basics.indexing.html), for when you need the exact behaviour of -1 or None.

---

16. Person to Follow / Role Model

Travis Oliphant (mapped by D.World for U06).

• Verified facts (checked 15 Oct 2026): NumPy's "About Us" page lists him as "project founder, 2005-2012". The same page says NumPy was created in 2005, building on the earlier Numeric and Numarray libraries.
• Why D.World recommends him: the shape rules you used today, which let small arrays combine with large ones without loops, are part of the library he founded. The same rules underpin pandas, which you start in U07.
• Honest note: we cite only what the NumPy page states. NumPy is now developed by a broad community, and his current roles change over time.
• Try this: reread the NumPy "About Us" page. In one sentence, name one project or field that it says depends on NumPy.

---

17. Glossary

Term	Meaning in this lesson
Broadcasting	NumPy's rule for combining arrays of different shapes, by stretching dimensions of size 1 (or missing dimensions) without copying
Compatible dimensions	Two dimensions, compared from the right, that are equal or where one is 1
Row vector	Shape (1, n), or a plain (n,); pairs with columns
Column vector	Shape (m, 1); pairs with rows
New axis	A length-1 dimension added with None / np.newaxis, e.g. cost[:, None] → (3, 1)
Reshape	Rearranging the same values into a new shape with the same total size; -1 = "work it out"
Transpose	Swapping rows and columns with .T; (3, 7) → (7, 3)
Silent mis-broadcasting	A broadcast that raises no error but pairs values incorrectly, e.g. (3,) with (3, 1) → (3, 3)

Review: shape and axis (L21, L23) · ValueError and reading tracebacks (L16).

---

18. Reflection

1. Explain the broadcasting rule to a classmate in two sentences without using code.
2. Which is more dangerous for a report, a loud shape error or a silent one? Why?
3. Look back at the worked example. What did you do, or what would you do, when the screen and your hand check disagreed?

---

19. Next Step

U06 is finished. You can now create arrays, select from them, compute along an axis, and combine different shapes safely. Next:

1. U06 Exit Check — Pickup-Point Sales Grid. An independent task that brings L21–L24 together. See the separate brief, U06_EXIT_CHECK_BRIEF.md.
2. U07 — Pandas Foundations, starting with L25. Each DataFrame column is a labelled 1-D array. pandas pairs values by label rather than by position, which removes many of the silent pairing mistakes you met today. L27 contrasts the two approaches.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Shape-prediction knowledge check (Q1–Q8)	Applies the broadcasting rule; spots silent mis-broadcasting; chooses sound evidence	7 or more of 8 correct
Debugging task with written justification (Independent Practice, Part 2)	Diagnoses an unfamiliar shape error; chooses the fix that matches the business meaning	Rubric L24-IP
Guided practice	Fixes a shape error with reshape and a new axis; applies per-day and per-point vectors; checks cells by hand	Rubric L24-GP
Independent Part 1, workplace task, mini challenge	Predicts compatibility for five pairs; builds a tested promotion check; finds a silent square-grid bug	Rubrics L24-IP, L24-WT, L24-MC

This lesson completes the teaching content of U06. The U06 Exit Check consolidates L21–L24.

---

References

1. NumPy Developers. Broadcasting (NumPy user guide). https://numpy.org/doc/stable/user/basics.broadcasting.html (opened 15 Oct 2026, N-BC). Supports: the general broadcasting rule (dimensions compared from the trailing end are compatible when equal or one is 1); missing leading dimensions; the result shape; the ValueError for incompatible shapes; scalar broadcasting described as the simplest case, with no copy made.
2. NumPy Developers. NumPy quickstart (v2.5 manual). https://numpy.org/doc/stable/user/quickstart.html (R03). Supports: reshape and -1; .T; broadcasting overview.
3. NumPy Developers. Indexing on ndarrays. https://numpy.org/doc/stable/user/basics.indexing.html (N-IDX). Supports: np.newaxis is an alias for None and adds a length-1 dimension.
4. NumPy Developers. About Us. https://numpy.org/about/ (NP-ABOUT). Supports the role-model facts.
5. D.World Stage 2 Detailed Lesson Map v1.0 (L24 row; U06 exit criterion). Internal.

U06 Exit Check — Pickup-Point Sales Grid

Learner brief. D.World · Stage 2 · U06 NumPy Foundations · Version 1 · 15 October 2026. Cedar Lane is a fictional business and all data is synthetic. This brief contains no answers.

---

1. What this is

This is an independent task that brings together everything in U06:

• arrays and shapes (L21);
• indexing and masks (L22);
• aggregation along an axis (L23);
• broadcasting (L24).

Assessment level	Application
Estimated effort	About 1.5 hours (planning estimate, unvalidated)
Timing	Self-paced. This is a suggested checkpoint (suggested challenge 1 of 6 in the six-week Stage 2 cadence), not a timed unlock.
Help allowed	Your own U06 notes and lessons, and the NumPy documentation. AI is optional and limited (see Section 6). The work and the reasoning must be yours.
How it is judged	Against a rubric (criteria in Section 7). Feedback comes after submission.

---

2. The situation

Cedar Lane now has four pickup points: North, Central, Harbor and Station. The manager has two weeks of notebook sales. They want a short, trustworthy analysis before the monthly review:

"Which points and days are strong or weak? Where are we selling very little? And what gross profit did each point make, given that each point has its own unit cost and we ran a promotion on two days?"

---

3. Files you receive

File	What it is
U06_EXIT_CHECK_BRIEF.md	This brief
u06_exit_data.py	The data. Do not edit it.
U06_exit_starter.ipynb	Starter notebook with the task headings and empty cells (preferred route)
U06_exit_script_template.py	Script template, only if D.World Support has recorded a Jupyter exception (map decision D04)
U06_exit_note_template.md	Prompts for your 100–200-word note

Keep all files in one folder, so that from u06_exit_data import ... works.

The data:

Name	Shape	Meaning
units	(4, 14)	Notebooks sold. Rows = pickup points (in the order of points); columns = days (in the order of days)
points	(4,)	North, Central, Harbor, Station
days	(14,)	W1-Mon … W1-Sun, W2-Mon … W2-Sun
price	(14,)	Selling price per notebook on each day (USD). It is lower on two promotion days
unit_cost	(4,)	Cost per notebook at each pickup point (USD)

---

4. Tasks

Do the tasks in order, in the starter notebook. For every calculation:

• write the expected shape first, in a comment or a Markdown cell;
• say in words which axis you collapse and why whenever you aggregate.

Task 1 — Load and check (setup).

1. Run a version cell, as in L21.
2. Import the data from u06_exit_data.py.
3. Assert the shapes of all five arrays.
4. In one sentence each, state what one row and one column of units mean.

Task 2 — Totals and means along stated axes.

1. Total units per pickup point over the two weeks.
2. Total units per day across all pickup points.
3. Mean daily units per pickup point.
4. Show that the totals from (1) and (2) add up to the same grand total.

For each of (1)–(3), assert the result's shape and write which axis you used.

Task 3 — Best and worst days.

1. For each pickup point, its best day and its worst day, as day labels.
2. For the business as a whole (all points together), the busiest day and the quietest day.
3. If two days tie, report the earlier day and mention the tie in a Markdown cell. Explain how you checked for ties.

Task 4 — Flag low-sales cells with a Boolean mask. The manager defines a low-sales cell as fewer than 14 units.

1. Create the mask and assert its shape.
2. Count the low-sales cells per pickup point and in total.
3. List Harbor's low-sales days as day labels.
4. Look at the data. Is there a value that looks unusual and that you would ask the manager about before drawing conclusions? Say which one, and what you would ask.

Task 5 — Revenue, cost and gross profit using broadcasting.

1. Revenue grid: units × the price of that day.
2. Cost grid: units × the unit cost of that pickup point.
3. Gross profit per pickup point over the two weeks, and gross margin per point (%, 1 decimal, for display only).
4. Before each broadcast, print both shapes. After it, assert the result shape. Check one inner cell of the cost grid by hand. Choose one that is not in the first row or the first column.

Task 6 — Fix the supplied shape error. A colleague drafted this cell. It is also in the starter notebook, marked "colleague's draft":

# Colleague's draft - intended: the cost of every cell, using that point's own unit cost
cost_grid_draft = units * unit_cost

1. Run it and copy the last line of the error.
2. Explain, using the broadcasting rule, which dimensions clash and why.
3. Fix it so that each value pairs as intended. Assert the shape.
4. Show that the fixed result matches your Task 5 cost grid.
5. Explain why at least one other "fix" that would make the error disappear is wrong. Examples: slicing the data, transposing only once, or repeating the costs.

Task 7 — Independent cross-checks in pure Python. Choose at least two of your results, for example one point's total and one point's gross profit. Recalculate each with plain Python loops on lists, without NumPy. Use units.tolist(), price.tolist() and unit_cost.tolist() to get lists. Assert that each matches your NumPy result; use a tolerance for decimals. Say which results you cross-checked.

Task 8 — Your note (100–200 words). Write it in the final Markdown cell, using U06_exit_note_template.md. It must:

• explain each axis choice you made (Tasks 2, 3, 4 and 5);
• state the main finding for the manager;
• name one limitation of this data or analysis.

---

5. How to submit

1. In JupyterLab, use Kernel → Restart Kernel and Run All Cells…. Every cell must run from top to bottom. The only error allowed is in the cell where you demonstrate the colleague's original error, and it must be caught with try/except ValueError, so the notebook still runs to the end.
2. Save the notebook as U06_exit_<yourname>.ipynb. Use your D.World learner name; do not put other personal details in the notebook.
3. Check that your note has 100–200 words.
4. Submit the .ipynb file for review as described on the unit page (members email it to D.World Support). You do not need to submit the data file.
5. Script route (only with a recorded exception): submit U06_exit_<yourname>.py, built from the script template. It must run from a fresh terminal (python U06_exit_<yourname>.py) without errors and print every required result. Submit the note as a separate .md or .txt file.

Before you submit, check:

• [ ] The version cell shows Python 3.13 or 3.14 and numpy 2.5.x (or a hosted Jupyter environment)
• [ ] Shape assertions for the data and for every result in Tasks 2–6
• [ ] An axis explanation for every aggregation
• [ ] One hand-checked inner cell (Task 5)
• [ ] The colleague's error demonstrated, explained and fixed (Task 6)
• [ ] At least two pure-Python cross-checks that pass (Task 7)
• [ ] A 100–200-word note (Task 8)
• [ ] Restart Kernel and Run All Cells completes

---

6. AI use (optional, bounded)

You may use an AI assistant only to:

• explain an error message;
• explain a NumPy function you already met in U06.

If you do:

• paste only code from this check or synthetic examples;
• in a Markdown cell, record what you asked, what it said, and how you verified it;
• do not ask an AI to write your solution or your note. Work you cannot explain will be treated as not yet demonstrated.

---

7. How your work is judged

Each criterion is marked 2 = meets, 1 = partly or 0 = not yet.

#	Criterion	What "meets" looks like
C1	Setup and reproducibility	Version cell; data shapes asserted; the notebook reruns cleanly from a restart
C2	Aggregation along stated axes	Correct totals and means, each with the correct axis, a shape assertion and the grand totals reconciled
C3	Best and worst days	Correct days per point and for the business, with ties handled and explained
C4	Boolean mask	Correct mask, counts and Harbor's days, plus a sensible question about an unusual value
C5	Broadcasting	Revenue and cost grids pair the right values (shapes printed and asserted; inner cell checked by hand); gross profit and margin per point correct
C6	Shape-error fix	Error explained with the rule; fix pairs values as intended; matches Task 5; a wrong alternative explained
C7	Independent cross-checks	At least two pure-Python cross-checks that pass
C8	Note	100–200 words; every axis choice explained; main finding; one limitation

To pass:

• Required gates. C1, C5, C6 and C7 must each be at least 1. The work must contain shape assertions and at least two passing pure-Python cross-checks, and the notebook must rerun cleanly.
• Total. At least 12 of 16 points.

If you do not pass yet, you get feedback and can revise and resubmit. This is not a timed or one-attempt test.

L25 — Series & DataFrames: Load and Inspect

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U07 Production v1 · Content date: 16 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13, pandas 3.0 and NumPy 2.5; openpyxl 3.1.5 where used) on Linux and on Windows 11 on 4 November 2026 (see the Code Test Report).

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	Series & DataFrames: Load and Inspect
Stage	Stage 2 — Data Handling
Unit	U07 — Pandas Foundations
Lesson ID	L25 (first lesson of U07)
Skills	Series; DataFrame; loading a CSV; inspecting a table
Difficulty	Beginner–intermediate
Estimated time	About 2 hours (planning estimate, unvalidated)
Assessment level	Practice
Prerequisites	U06 (arrays, shape, dtype, boolean masks). From U04: dicts (L15) and functions (L16).

Time breakdown (planning estimate):

• Concept and worked example: 45 min.
• Guided practice: 20 min.
• Independent practice: 20 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 10 min (optional).
• Workplace task and mini challenge: 15 min.

What you need:

• Your dworld-env environment from L21, or your hosted Jupyter environment. pandas was installed in L21; you do not need to install anything new.
• The practice_files folder from the U07 learner bundle. It contains L25_starter.ipynb and a u07_data folder with the CSV files.
• Start JupyterLab from inside practice_files, or open the notebook that lives there. Section 4, Part D explains why.

---

2. Learning Objective

By the end of this lesson, you will be able to:

• load a CSV file into a pandas DataFrame;
• report its row and column counts, column names, dtypes, first and last rows and summary statistics;
• end your inspection with one written question that the inspection raises.

D.World approved objective (verbatim): Load a CSV file into a DataFrame and report its row and column counts, column names, dtypes, first and last rows and summary statistics, ending with one written question the inspection raises.

---

3. Why It Matters

In U06 you worked with grids of numbers. Real business data rarely looks like that. An order export has text (order IDs, region codes, statuses), dates and numbers side by side, and every column has a name.

pandas is the Python library for this kind of table. Its two main objects are:

• the Series, a labelled one-dimensional array (one column);
• the DataFrame, a table of Series that share the same row labels.

Before you calculate anything from a new file, you need to know what is actually in it. How many rows? Which columns? Did each column load as the type you expect? Are there values you did not expect? Analysts who skip this step build reports on files they have never looked at, and their first mistake is usually a wrong row count or a "number" column that is really text.

Key principle: inspect before you calculate. Every inspection ends with a written question, because a good inspection always finds something to ask.

---

4. Concept

Part A — A Series is a labelled array

import numpy as np
import pandas as pd

units = pd.Series([117, 168, 72], index=["North", "Central", "Harbor"], name="units")
print(units)
print(units["Central"])

North      117
Central    168
Harbor      72
Name: units, dtype: int64
168

A Series has:

• values (here 117, 168, 72), like a NumPy array;
• an index: one label per value (here the pickup-point names);
• a dtype (here int64) and, optionally, a name.

You can look values up by label (units["Central"]) rather than by position. In U06 you had to remember that row 1 was Central. Here the label travels with the value.

Part B — A DataFrame is a table of Series

You can build a small DataFrame from a dict (L15). Each key becomes a column label. Each list becomes a column.

points = pd.DataFrame({
    "point": ["North", "Central", "Harbor"],
    "units": [117, 168, 72],
    "unit_cost": [5.00, 4.80, 5.40],
})
print(points)
print(points.shape)
print(list(points.columns))

     point  units  unit_cost
0    North    117        5.0
1  Central    168        4.8
2   Harbor     72        5.4
(3, 3)
['point', 'units', 'unit_cost']

• The numbers on the left (0, 1, 2) are the index, the row labels. pandas creates 0, 1, 2, … when you do not give one.
• .shape is (rows, columns), exactly as in NumPy.
• .columns holds the column labels.

One column of a DataFrame is a Series:

col = points["units"]
print(type(col).__name__)
print(col.dtype)

Series
int64

Part C — The link to NumPy

Underneath, each numeric column holds a NumPy-style array. .to_numpy() gives you that array, so everything from U06 still works:

arr = points["units"].to_numpy()
print(type(arr).__name__, arr.shape, arr.dtype)
print(arr.sum())

ndarray (3,) int64
357

The important difference: NumPy pairs values by position; pandas pairs values by label. You will see why that matters in L27.

Part D — File paths and the working directory (brief)

When Python opens a file, it needs a path:

• an absolute path starts from the top of the disk, for example C:\Users\amal\... or /home/amal/.... It works only on that computer;
• a relative path, such as u07_data/orders.csv, is read starting from the working directory. That is the folder Python is "standing in".

In Jupyter, the working directory is normally the folder that contains the notebook. That is why the notebooks and the u07_data folder sit side by side in practice_files. You can check where you are, and whether a file can be found:

from pathlib import Path

print(Path("u07_data/orders.csv").exists())

True

If this prints False, Python is standing in a different folder. Run print(Path.cwd()) to see which one. Then either move the notebook into practice_files or restart JupyterLab from there. Do not paste an absolute path from your own computer into shared work: a colleague's computer will not have that folder.

Part E — Loading and the inspection checklist

pd.read_csv reads a CSV file into a DataFrame. In U07 we use it with its defaults: commas between values, the first line is the header. Other delimiters, encodings and Excel files are U08.

lines = pd.read_csv("u07_data/order_lines.csv")
print(lines.shape)
print(list(lines.columns))

(384, 7)
['order_id', 'line_no', 'order_date', 'region_code', 'product_id', 'quantity', 'unit_price']

The inspection checklist used in all of U07:

Step	Code	What you are looking for
1. Size	.shape	Rows and columns. Compare rows with the file's line count minus the header
2. Names	.columns	Exact column labels; spelling and spaces matter
3. Look	.head(), .tail()	First and last rows; does the content match the names?
4. Types	.dtypes or .info()	Does each column have the type you expect? .info() also shows non-null counts
5. Numbers	.describe()	Count, mean, min, max of numeric columns; anything impossible?
6. Categories	["col"].value_counts()	Which labels appear, and how often?
7. Question	(you write it)	One question the inspection raises

print(lines.head(3))
print(lines.tail(2))

  order_id  line_no  order_date region_code product_id  quantity  unit_price
0   O10056        1  2026-08-01         HAR        P07         1        35.0
1   O10056        2  2026-08-01         HAR        P08         2         6.0
2   O10138        1  2026-08-01         NOR        P03         2         3.6
    order_id  line_no  order_date region_code product_id  quantity  unit_price
382   O10150        3  2026-10-30         STA        P02         2        15.0
383   O10180        1  2026-10-30         CEN        P01         2        12.0

lines.info()

<class 'pandas.DataFrame'>
RangeIndex: 384 entries, 0 to 383
Data columns (total 7 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   order_id     384 non-null    str    
 1   line_no      384 non-null    int64  
 2   order_date   384 non-null    str    
 3   region_code  384 non-null    str    
 4   product_id   384 non-null    str    
 5   quantity     384 non-null    int64  
 6   unit_price   384 non-null    float64
dtypes: float64(1), int64(2), str(4)
memory usage: 21.1 KB

.info() prints directly, so you do not wrap it in print(). The "memory usage" line may differ slightly on your computer.

print(lines.describe())

          line_no    quantity  unit_price
count  384.000000  384.000000  384.000000
mean     1.697917    2.554688   17.454948
std      0.865586    1.946310   11.377988
min      1.000000    1.000000    3.600000
25%      1.000000    1.000000    8.000000
50%      1.000000    2.000000   15.000000
75%      2.000000    3.000000   24.000000
max      5.000000   10.000000   42.000000

.describe() summarizes numeric columns only by default. Notice that line_no is included because it is stored as a number, even though its mean (1.70) means little. An inspection is also about deciding which numbers make sense to summarize.

print(lines["region_code"].value_counts())

region_code
CEN    108
STA     77
NOR     68
WEB     67
HAR     64
Name: count, dtype: int64

Part F — pandas 3 and the string dtype

Look at the Dtype column of .info() again. Text columns show str. In pandas 3, text is stored in a dedicated string dtype by default. Older tutorials and older pandas versions show object for the same columns. When you read those tutorials, treat object and str as "this column holds text".

print(lines["product_id"].dtype)
print(lines["order_date"].dtype)

str
str

order_date looks like a date to you, but pandas read it as text. read_csv with its defaults does not guess dates. For now, notice it and write it down:

• setting the right type while loading is U08 (L30);
• repairing types after loading is U09 (L37).

A text column of ISO dates (2026-08-01) still sorts correctly, because the year comes first. That will not be true for every date format.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. Before the weekly sales review, the analyst receives order_lines.csv. Each row is one product line of one order. The manager asks: "Is this the complete export for August to October?"

Step 1 — Expected size first. Count the file's lines without pandas, so that you have an independent number to compare with:

with open("u07_data/order_lines.csv", encoding="utf-8") as f:
    n_file_lines = sum(1 for _ in f)
print(n_file_lines, "lines in the file, so", n_file_lines - 1, "data rows")

385 lines in the file, so 384 data rows

Step 2 — Load and compare.

lines = pd.read_csv("u07_data/order_lines.csv")
assert lines.shape[0] == n_file_lines - 1
print(lines.shape)

(384, 7)

✔ The row count matches the file.

Step 3 — Date range. Because the dates are ISO text, min and max work:

print(lines["order_date"].min(), "to", lines["order_date"].max())
print(lines["order_id"].nunique(), "distinct orders")

2026-08-01 to 2026-10-30
200 distinct orders

The export runs from 1 August to 30 October 2026. There are 384 lines but only 200 distinct orders, because an order can have several lines.

Step 4 — Recompute one describe() value with NumPy. Do not take a summary on trust. Recompute the mean quantity from the raw array:

q = lines["quantity"].to_numpy()
print(q.mean())
print(lines["quantity"].describe()["mean"])
assert np.isclose(q.mean(), lines["quantity"].describe()["mean"])

2.5546875
2.5546875

✔

Step 5 — Write the inspection summary (five lines plus a question).

1. order_lines.csv has 384 rows and 7 columns; the file has 385 lines, so the header plus 384 data rows — complete.
2. Columns: order_id, line_no, order_date, region_code, product_id, quantity, unit_price.
3. Dates run from 2026-08-01 to 2026-10-30; 200 distinct orders, so about 1.9 lines per order.
4. Types: quantity and line_no are integers, unit_price is a float; order_date loaded as text (str), not as a date.
5. Quantity ranges from 1 to 10; unit_price from 3.60 to 42.00; no missing values in any column (.info() shows 384 non-null everywhere).

Question: the minimum unit_price is 3.60, but is there a product whose list price is that low, or is 3.60 a discounted price? I need the product table to check.

When a check fails (for example, the row count does not match the file), stop. Do not continue to calculations. Open the file in a text editor, look at the end, and write down what you see.

---

6. Visual Explanation

            column labels  →   order_id  line_no  order_date  region_code  product_id  quantity  unit_price
  index  ┌─────────────────┬──────────┬────────┬────────────┬────────────┬───────────┬─────────┬───────────┐
  (row   │ 0               │ O10056   │   1    │ 2026-08-01 │    HAR     │    P07    │    1    │   35.00   │
  labels)│ 1               │ O10056   │   2    │ 2026-08-01 │    HAR     │    P08    │    2    │    6.00   │
    ↓    │ …               │   …      │   …    │     …      │     …      │     …     │    …    │     …     │
         └─────────────────┴──────────┴────────┴────────────┴────────────┴───────────┴─────────┴───────────┘
                              str       int64       str          str          str       int64     float64
                                                    ▲ looks like a date, loaded as text
  one column  = a Series (values + the same index)          .shape = (384, 7)

Alt text: A table diagram of the order-lines DataFrame. Across the top are the seven column labels: order_id, line_no, order_date, region_code, product_id, quantity and unit_price. Down the left side is the index, the row labels 0, 1 and so on. Two example rows are shown. Under each column is its dtype: str for order_id, int64 for line_no, str for order_date, str for region_code, str for product_id, int64 for quantity and float64 for unit_price. An arrow marks order_date: it looks like a date but was loaded as text. A note says that one column is a Series, made of values plus the same index, and that the shape is 384 rows by 7 columns.

---

7. Guided Practice

Open L25_starter.ipynb in practice_files. You will inspect u07_data/orders.csv: one row per order. Work through the checklist from Part E. Write what you expect before each step.

1. Expected size. Count the lines of orders.csv without pandas, as in the worked example.
2. Load. Read the file into a DataFrame called orders. Assert that the number of rows equals the line count minus 1.
3. Names and a look. Print the column labels, the first 5 rows and the last 3 rows.
4. Types. Run orders.info(). Which columns are text? Which are numbers? Is any column's type surprising?
5. Numbers. Print orders.describe(). Recompute the mean order_value with NumPy and check it with np.isclose.
6. Categories. Print value_counts() for status and for region_code. Check that each set of counts adds up to the number of rows.
7. Summary. Write five lines and one question, as in Step 5 of the worked example.
8. Run Restart Kernel and Run All Cells… and confirm that every cell runs without an error.

---

8. Independent Practice

Load u07_data/products.csv (the product master table). Write an inspection summary in a Markdown cell of your own notebook, L25_products.ipynb:

1. Rows and columns, checked against the file's line count.
2. Column names.
3. Dtypes, with a comment on any column whose dtype looks wrong for its meaning. Do not fix it: fixing types is U09.
4. Which column is the likely key, the column that identifies each row uniquely? Give your evidence: compare nunique() with the number of rows.
5. One summary-statistic observation (for example, the range of unit_cost) and one categorical observation (value_counts() of a text column).
6. One question the inspection raises.

---

9. Knowledge Check — Inspection

Q1. df.shape returns (200, 7). What does that mean?

• A) 200 columns and 7 rows
• B) 200 rows and 7 columns
• C) 200 cells in total
• D) 7 missing values

Q2. A CSV file has 51 lines, including the header. After pd.read_csv with defaults, how many rows should the DataFrame have?

• A) 50
• B) 51
• C) 52
• D) It cannot be known

Q3. Which statement about a Series is correct?

• A) It is a two-dimensional table
• B) It is a one-dimensional array of values with an index of labels
• C) It can only hold text
• D) It has no dtype

Q4. In pandas 3, a text column loaded by read_csv usually shows which dtype?

• A) int64
• B) float64
• C) str
• D) datetime64

Q5. orders.describe() does not show the status column. Why?

• A) The column is empty
• B) describe() summarizes numeric columns by default, and status is text
• C) The column name is misspelled
• D) describe() only shows the first five columns

Q6. Your notebook says FileNotFoundError: ... 'u07_data/orders.csv', but the file exists on your computer. What is the most likely cause?

• A) The file is too large
• B) pandas cannot read CSV files
• C) The working directory is not the folder that contains u07_data
• D) The file must be opened in Excel first

Q7. Which code gives the number of distinct order IDs?

• A) lines["order_id"].count()
• B) len(lines)
• C) lines["order_id"].nunique()
• D) lines.shape[1]

Q8. A column called boxes_counted loads with dtype str instead of a number. What is the best first action in L25?

• A) Delete the column
• B) Write it down in the inspection summary and look at its value_counts() to see which values are not numbers
• C) Change every value to 0
• D) Ignore it, because describe() will handle it

---

10. AI in Practice (optional, bounded)

Goal: ask an AI assistant to explain an .info() output, then check every number it gives you against your own inspection.

1. Paste only the .info() output from your guided practice (orders.info()). It is synthetic. Ask:

"I am learning pandas 3. Here is the .info() output of a synthetic order table. Explain each part. How many rows and columns are there? Are there missing values? Which columns are text? Is any type surprising for what the column name suggests?"

2. Before reading the answer, write your own numbers: rows, columns, missing values per column, and which columns are text.
3. Compare, using Section 11.

Boundaries:

• AI is optional.
• Paste only synthetic output like this. Never paste real customer data or real exports.
• AI explanations of older pandas versions often say object for text columns; in pandas 3 you see str.

---

11. AI Verification

1. Rows and columns: the AI's numbers must equal orders.shape. If its row count is one more than orders.shape[0], it probably counted the header.
2. Missing values: compare its claim with the Non-Null Count column and with orders.isna().sum().
3. Text columns: check them against orders.dtypes.
4. Surprising type: did it comment on any column whose type does not match what its name suggests? Compare with your own answer to guided step 4.
5. Reject invented details. If the AI mentions a column or a value that is not in your output, it is guessing. Write down what was wrong.

---

12. Business Lens

What did we actually receive? Every report starts with a file someone else produced: an export from a shop system, a spreadsheet from a colleague, a download from a supplier. Files are often not what the request said:

• one month is missing, or one day is included twice;
• a "number" column contains a value such as 1,200 (with a thousands comma) or approx 40, so it loads as text;
• the region codes are not the ones on the sales map.

A five-line inspection summary takes ten minutes. It protects every number that follows. It also gives you a professional way to start the conversation with whoever sent the file: "The file has [number] orders from [first date] to [last date], and [number] are marked cancelled. Should cancelled orders count in revenue?" That question is worth more than a confident total built on an unchecked file.

---

13. Workplace Task — October Export Check

Fictional business: Cedar Lane. Your manager writes:

"Before Monday's review, can you check the October order file the shop system sent? I just need to know it's the full month and whether anything looks odd."

In L25_october_check.ipynb:

1. Load u07_data/orders_2026_10.csv. Check the row count against the file's line count.
2. Report the first and last order date and the number of distinct orders and customers.
3. Report the status counts and the region_code counts.
4. Recompute the total of order_value with NumPy and compare it with orders_oct["order_value"].sum().
5. Write a short message to the manager (four to six sentences): what the file contains, whether it looks complete for October, one thing that looks odd or needs a decision, and one question.
6. Run Restart Kernel and Run All Cells….

---

14. Mini Challenge — An Unseen File

Fictional business: Hilltop Honey, a separate business that sells jars of honey at two market stalls. You have never seen their file before: u07_data/hilltop_stalls.csv.

There is no checklist for this one. Inspect the file in whatever way you think is right, and write a five-line inspection summary plus one question, as you would for a colleague.

Your summary must mention anything about the types that would stop you calculating total revenue straight away. Do not fix anything; describe what you found and what you would ask.

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 25 min):

• 10 minutes to pandas (pandas 3.0 user guide): the sections "Object creation" and "Viewing data". https://pandas.pydata.org/docs/user_guide/10min.html (Opened 16 Oct 2026.)

Recommended:

• Python for Data Analysis, 3rd edition, Wes McKinney, Ch. 5 "Getting Started with pandas", sections 5.1 (Series and DataFrame) and 5.3 (summarizing and computing descriptive statistics). Free to read online: https://wesmckinney.com/book/pandas-basics.html. The book was written for an earlier pandas version, so text columns appear as object there.

Reference (look things up when needed):

• pandas: IO tools, the read_csv section. You only need the defaults this week; U08 uses the options. https://pandas.pydata.org/docs/user_guide/io.html

---

16. Person to Follow / Role Model

Wes McKinney (mapped by D.World for U07).

• Verified facts (checked 16 Oct 2026): in his book Python for Data Analysis, Ch. 1, he writes that he started building pandas in early 2008 while working at AQR Capital Management. The pandas "About" page says pandas was open-sourced by the end of 2009. He is the author of Python for Data Analysis.
• Why D.World recommends him: he created the library this whole unit uses, and his book (Ch. 5, recommended above) explains Series and DataFrames with small, clear examples.
• Honest note: we cite only what his book and the pandas site state. pandas is now developed by a large community, and his current roles change over time.
• Try this: open the pandas "About" page (https://pandas.pydata.org/about/) and write one sentence about how the project is supported today.

---

17. Glossary

Term	Meaning in this lesson
Series	A one-dimensional labelled array: values plus an index
DataFrame	A table of columns (Series) that share the same index
Index	The row labels of a Series or DataFrame; 0, 1, 2, … by default
Column label	The name of a DataFrame column, e.g. "order_value"
pandas dtype	The data type of a column, e.g. int64, float64, str
String dtype	pandas 3's default type for text columns, shown as str (older versions show object)
File path	The location of a file, written as folders and a file name
Relative path	A path read starting from the working directory, e.g. u07_data/orders.csv
Working directory	The folder Python is currently "standing in"; in Jupyter, usually the notebook's folder

Review: dict (L15) · array, shape, dtype (L21).

---

18. Reflection

1. In one sentence each, explain the difference between a Series and a DataFrame, and between an index and a column.
2. Which step of the inspection checklist found the most useful information today? Why?
3. Write one question about Cedar Lane's data that you cannot answer yet but expect to answer by the end of U07.

---

19. Next Step

You can now load a table and describe what is in it. Next:

• L26 — Selecting, Filtering & Sorting. You will pick out the rows and columns that answer a business question, for example "all delayed orders above a value threshold", and report how many rows the answer has.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Completed inspection checklist (Guided Practice, orders.csv)	Loads a CSV, checks the row count independently, reads types, summary statistics and categories	Rubric L25-GP
Written inspection summary (Independent Practice, products.csv)	Reports size, names, types, the likely key with evidence, and a question	Rubric L25-IP
Knowledge check (Q1–Q8)	Reads shapes and dtypes; understands relative paths; chooses sound first actions	7 or more of 8 correct
Workplace task and mini challenge	Writes a manager-ready check message; inspects an unseen file without a checklist	Rubrics L25-WT, L25-MC

---

References

1. pandas development team. 10 minutes to pandas (pandas 3.0 user guide). https://pandas.pydata.org/docs/user_guide/10min.html (R04). Supports: Series and DataFrame creation, head, tail, describe, to_numpy.
2. pandas development team. IO tools (text, CSV, HDF5, …). https://pandas.pydata.org/docs/user_guide/io.html (P-IO). Supports: read_csv defaults (comma delimiter, first line as header).
3. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 5 "Getting Started with pandas". https://wesmckinney.com/book/pandas-basics.html (WM-CH5). Supports: Series, DataFrame, index, descriptive statistics.
4. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 1 "Preliminaries", section on pandas. https://wesmckinney.com/book/preliminaries.html (WM-CH1). Supports the role-model fact about 2008 and AQR Capital Management.
5. pandas development team. About pandas. https://pandas.pydata.org/about/ (P-ABOUT). Supports: open-sourced by the end of 2009.
6. D.World Stage 2 Detailed Lesson Map v1.0 (L25 row). Internal.

L25 — Series وDataFrame: التحميل والفحص

ملف المتعلّم. الإجابات والحلول النموذجية في ملف منفصل خاص بالمدرّس فقط. الإصدار: U07 Production v1 · تاريخ المحتوى: 16 أكتوبر 2026. تقديرات الوقت تقديرات للتخطيط إلى أن يختبرها المتعلّمون. الكود في هذا الدرس شُغّل باستخدام تجهيزات الدورة من L21 (Python 3.13 وpandas 3.0 وNumPy 2.5؛ وopenpyxl 3.1.5 حيث استُخدمت) على Linux وعلى Windows 11 في 4 نوفمبر 2026 (راجع تقرير اختبار الكود Code Test Report).

كل الأنشطة التجارية خيالية، وكل البيانات مصطنعة.

---

1. بطاقة الدرس

البند	القيمة
العنوان	Series وDataFrame: التحميل والفحص
المرحلة	المرحلة 2 — التعامل مع البيانات
الوحدة	U07 — أساسيات pandas
رقم الدرس	L25 (أول درس في U07)
المهارات	Series؛ DataFrame؛ تحميل ملف CSV؛ فحص جدول
المستوى	مبتدئ إلى متوسط
الوقت المتوقع	حوالي ساعتين (تقدير للتخطيط، غير مُتحقَّق منه)
مستوى التقييم	تدريب
المتطلبات السابقة	U06 (المصفوفات، والشكل shape، ونوع البيانات dtype، والأقنعة المنطقية boolean masks). ومن U04: القواميس (L15) والدوال (L16).

تفصيل الوقت (تقدير للتخطيط):

• المفهوم والمثال المحلول: 45 دقيقة.
• التدريب الموجَّه: 20 دقيقة.
• التدريب المستقل: 20 دقيقة.
• اختبار الفهم: 10 دقائق.
• الذكاء الاصطناعي في التطبيق والتحقق منه: 10 دقائق (اختياري).
• مهمة العمل والتحدي القصير: 15 دقيقة.

ما تحتاج إليه:

• بيئتك dworld-env من L21، أو بيئة Jupyter المستضافة. ثُبّتت pandas في L21، فلا تحتاج إلى تثبيت أي شيء جديد.
• مجلد practice_files من حزمة المتعلّم الخاصة بـ U07. وفيه L25_starter.ipynb ومجلد u07_data الذي يضم ملفات CSV.
• شغّل JupyterLab من داخل practice_files، أو افتح الدفتر الموجود هناك. ويشرح الجزء D من القسم 4 السبب.

---

2. هدف التعلّم

في نهاية هذا الدرس ستكون قادرًا على أن:

• تحمّل ملف CSV في جدول بيانات pandas (DataFrame)؛
• تذكر عدد صفوفه وأعمدته، وأسماء الأعمدة، وأنواع البيانات (dtypes)، وأول الصفوف وآخرها، والإحصاءات الوصفية الملخِّصة؛
• تختم فحصك بسؤال مكتوب واحد يثيره هذا الفحص.

الهدف المعتمد من D.World (بنصّه): Load a CSV file into a DataFrame and report its row and column counts, column names, dtypes, first and last rows and summary statistics, ending with one written question the inspection raises.

---

3. لماذا يهمّك هذا؟

في U06 تعاملت مع شبكات من الأرقام. لكن بيانات الأعمال الحقيقية نادرًا ما تبدو هكذا. ففي ملف تصدير الطلبات تجد نصوصًا (أرقام الطلبات، ورموز المناطق، والحالات) وتواريخ وأرقامًا جنبًا إلى جنب، ولكل عمود اسم.

ومكتبة pandas هي مكتبة Python المخصّصة لهذا النوع من الجداول. وكائناها الرئيسيان هما:

• Series: مصفوفة أحادية البعد ذات تسميات (عمود واحد)؛
• DataFrame: جدول من عدة Series تشترك في تسميات الصفوف نفسها.

وقبل أن تحسب أي شيء من ملف جديد، عليك أن تعرف ما الموجود فيه فعلًا. كم صفًا فيه؟ ما أعمدته؟ هل حُمّل كل عمود بالنوع الذي تتوقعه؟ هل توجد قيم لم تتوقعها؟ المحللون الذين يتخطّون هذه الخطوة يبنون تقارير على ملفات لم ينظروا فيها قط، وأول أخطائهم عادةً عدد صفوف خاطئ أو عمود "أرقام" هو في الحقيقة نص.

المبدأ الأساسي: افحص قبل أن تحسب. وكل فحص ينتهي بسؤال مكتوب، لأن الفحص الجيد يجد دائمًا شيئًا يستحق السؤال.

---

4. المفهوم

الجزء A — الـ Series مصفوفة ذات تسميات

import numpy as np
import pandas as pd

units = pd.Series([117, 168, 72], index=["North", "Central", "Harbor"], name="units")
print(units)
print(units["Central"])

North      117
Central    168
Harbor      72
Name: units, dtype: int64
168

للـ Series:

• قيم (هنا 117 و168 و72)، مثل مصفوفة NumPy؛
• فهرس (index): تسمية واحدة لكل قيمة (هنا أسماء نقاط الاستلام)؛
• نوع بيانات (dtype) (هنا int64)، واسم (name) اختياري.

يمكنك البحث عن القيم بالتسمية (units["Central"]) بدلًا من الموضع. في U06 كان عليك أن تتذكر أن الصف 1 هو Central. أما هنا فالتسمية تنتقل مع القيمة.

الجزء B — الـ DataFrame جدول من عدة Series

يمكنك بناء DataFrame صغير من قاموس (dict) (L15). كل مفتاح يصبح تسمية عمود (column label)، وكل قائمة تصبح عمودًا.

points = pd.DataFrame({
    "point": ["North", "Central", "Harbor"],
    "units": [117, 168, 72],
    "unit_cost": [5.00, 4.80, 5.40],
})
print(points)
print(points.shape)
print(list(points.columns))

     point  units  unit_cost
0    North    117        5.0
1  Central    168        4.8
2   Harbor     72        5.4
(3, 3)
['point', 'units', 'unit_cost']

• الأرقام على اليسار (0 و1 و2) هي الفهرس، أي تسميات الصفوف. وتنشئ pandas التسميات 0، 1، 2، … إذا لم تعطها فهرسًا.
• .shape هو (rows, columns)، أي (الصفوف، الأعمدة)، تمامًا كما في NumPy.
• .columns يحتوي على تسميات الأعمدة.

والعمود الواحد من DataFrame هو Series:

col = points["units"]
print(type(col).__name__)
print(col.dtype)

Series
int64

الجزء C — الصلة بـ NumPy

في الداخل، يحمل كل عمود رقمي مصفوفة على طريقة NumPy. والدالة .to_numpy() تعطيك هذه المصفوفة، فيظل كل ما تعلّمته في U06 صالحًا:

arr = points["units"].to_numpy()
print(type(arr).__name__, arr.shape, arr.dtype)
print(arr.sum())

ndarray (3,) int64
357

الفرق المهم: NumPy تقرن القيم بحسب الموضع، أما pandas فتقرنها بحسب التسمية. وسترى لماذا يهم ذلك في L27.

الجزء D — مسارات الملفات ومجلد العمل (باختصار)

عندما يفتح Python ملفًا، فإنه يحتاج إلى مسار (path):

• المسار المطلق (absolute path) يبدأ من قمة القرص، مثل C:\Users\amal\... أو /home/amal/.... ولا يعمل إلا على ذلك الجهاز؛
• المسار النسبي (relative path)، مثل u07_data/orders.csv، يُقرأ بدءًا من مجلد العمل (working directory)، وهو المجلد الذي "يقف فيه" Python.

في Jupyter يكون مجلد العمل عادةً المجلد الذي يحتوي على الدفتر. ولهذا توجد الدفاتر ومجلد u07_data جنبًا إلى جنب في practice_files. ويمكنك أن تتحقق من مكانك، ومن إمكانية العثور على ملف:

from pathlib import Path

print(Path("u07_data/orders.csv").exists())

True

إذا طبع هذا False، فإن Python يقف في مجلد آخر. شغّل print(Path.cwd()) لترى أي مجلد هو. ثم انقل الدفتر إلى practice_files، أو أعد تشغيل JupyterLab من هناك. لا تلصق مسارًا مطلقًا من جهازك في عمل مشترك: فجهاز زميلك لن يكون فيه ذلك المجلد.

الجزء E — التحميل وقائمة الفحص

الدالة pd.read_csv تقرأ ملف CSV في DataFrame. وفي U07 نستخدمها بإعداداتها الافتراضية: فواصل (commas) بين القيم، والسطر الأول هو الترويسة (header). أما الفواصل الأخرى وترميزات النصوص (encodings) وملفات Excel فموضوعها U08.

lines = pd.read_csv("u07_data/order_lines.csv")
print(lines.shape)
print(list(lines.columns))

(384, 7)
['order_id', 'line_no', 'order_date', 'region_code', 'product_id', 'quantity', 'unit_price']

قائمة الفحص (inspection checklist) المستخدمة في U07 كلها:

الخطوة	الكود	ما الذي تبحث عنه
1. الحجم	.shape	الصفوف والأعمدة. قارن الصفوف بعدد أسطر الملف ناقص سطر الترويسة
2. الأسماء	.columns	تسميات الأعمدة بالضبط؛ فالتهجئة والمسافات مهمة
3. نظرة	.head()، .tail()	أول الصفوف وآخرها؛ هل يطابق المحتوى الأسماء؟
4. الأنواع	.dtypes أو .info()	هل لكل عمود النوع الذي تتوقعه؟ وتعرض .info() أيضًا عدد القيم غير الفارغة (non-null)
5. الأرقام	.describe()	العدد والمتوسط والحد الأدنى والأقصى للأعمدة الرقمية؛ هل يوجد شيء مستحيل؟
6. الفئات	["col"].value_counts()	ما التسميات التي تظهر، وكم مرة؟
7. السؤال	(تكتبه أنت)	سؤال واحد يثيره الفحص

print(lines.head(3))
print(lines.tail(2))

  order_id  line_no  order_date region_code product_id  quantity  unit_price
0   O10056        1  2026-08-01         HAR        P07         1        35.0
1   O10056        2  2026-08-01         HAR        P08         2         6.0
2   O10138        1  2026-08-01         NOR        P03         2         3.6
    order_id  line_no  order_date region_code product_id  quantity  unit_price
382   O10150        3  2026-10-30         STA        P02         2        15.0
383   O10180        1  2026-10-30         CEN        P01         2        12.0

lines.info()

<class 'pandas.DataFrame'>
RangeIndex: 384 entries, 0 to 383
Data columns (total 7 columns):
 #   Column       Non-Null Count  Dtype
---  ------       --------------  -----
 0   order_id     384 non-null    str
 1   line_no      384 non-null    int64
 2   order_date   384 non-null    str
 3   region_code  384 non-null    str
 4   product_id   384 non-null    str
 5   quantity     384 non-null    int64
 6   unit_price   384 non-null    float64
dtypes: float64(1), int64(2), str(4)
memory usage: 21.1 KB

الدالة .info() تطبع مباشرة، فلا تضعها داخل print(). وقد يختلف سطر "memory usage" (استهلاك الذاكرة) قليلًا على جهازك.

print(lines.describe())

          line_no    quantity  unit_price
count  384.000000  384.000000  384.000000
mean     1.697917    2.554688   17.454948
std      0.865586    1.946310   11.377988
min      1.000000    1.000000    3.600000
25%      1.000000    1.000000    8.000000
50%      1.000000    2.000000   15.000000
75%      2.000000    3.000000   24.000000
max      5.000000   10.000000   42.000000

الدالة .describe() تلخّص الأعمدة الرقمية فقط افتراضيًا. لاحظ أن line_no مُدرَج لأنه مخزَّن كرقم، مع أن متوسطه (1.70) لا يعني الكثير. فالفحص يعني أيضًا أن تقرر أي الأرقام يصحّ تلخيصها.

print(lines["region_code"].value_counts())

region_code
CEN    108
STA     77
NOR     68
WEB     67
HAR     64
Name: count, dtype: int64

الجزء F — pandas 3 ونوع البيانات النصي

انظر مرة أخرى إلى عمود Dtype في ناتج .info(). الأعمدة النصية تظهر بالنوع str. ففي pandas 3 تُخزَّن النصوص افتراضيًا في نوع بيانات نصي (string dtype) مخصّص لها. أما الدروس القديمة وإصدارات pandas الأقدم فتعرض object للأعمدة نفسها. وعندما تقرأ تلك الدروس، اعتبر object وstr بمعنى "هذا العمود يحمل نصًا".

print(lines["product_id"].dtype)
print(lines["order_date"].dtype)

str
str

يبدو لك order_date تاريخًا، لكن pandas قرأته نصًا. فالدالة read_csv بإعداداتها الافتراضية لا تخمّن التواريخ. والمطلوب الآن أن تلاحظ ذلك وتدوّنه:

• تحديد النوع الصحيح أثناء التحميل موضوعه U08 (L30)؛
• إصلاح الأنواع بعد التحميل موضوعه U09 (L37).

عمود نصي من تواريخ بصيغة ISO (2026-08-01) يُرتَّب ترتيبًا صحيحًا مع ذلك، لأن السنة تأتي أولًا. لكن هذا لن يصحّ مع كل صيغ التواريخ.

---

5. مثال محلول

Cedar Lane (نشاط خيالي)، بيانات مصطنعة. قبل المراجعة الأسبوعية للمبيعات، يتسلّم المحلل الملف order_lines.csv. كل صف فيه سطر منتج واحد من طلب واحد. ويسأل المدير: "هل هذا هو ملف التصدير الكامل للفترة من أغسطس إلى أكتوبر؟"

الخطوة 1 — الحجم المتوقع أولًا. عُدّ أسطر الملف من دون pandas، حتى يكون لديك رقم مستقل تقارن به:

with open("u07_data/order_lines.csv", encoding="utf-8") as f:
    n_file_lines = sum(1 for _ in f)
print(n_file_lines, "lines in the file, so", n_file_lines - 1, "data rows")

385 lines in the file, so 384 data rows

الخطوة 2 — حمّل وقارن.

lines = pd.read_csv("u07_data/order_lines.csv")
assert lines.shape[0] == n_file_lines - 1
print(lines.shape)

(384, 7)

✔ عدد الصفوف يطابق الملف.

الخطوة 3 — مدى التواريخ. لأن التواريخ نصوص بصيغة ISO، فإن min وmax تعملان:

print(lines["order_date"].min(), "to", lines["order_date"].max())
print(lines["order_id"].nunique(), "distinct orders")

2026-08-01 to 2026-10-30
200 distinct orders

يمتد ملف التصدير من 1 أغسطس إلى 30 أكتوبر 2026. وفيه 384 سطرًا لكن 200 طلب مختلف فقط، لأن الطلب الواحد قد يضم عدة أسطر.

الخطوة 4 — أعد حساب قيمة واحدة من describe() باستخدام NumPy. لا تأخذ الملخّص على الثقة. أعد حساب متوسط الكمية من المصفوفة الخام:

q = lines["quantity"].to_numpy()
print(q.mean())
print(lines["quantity"].describe()["mean"])
assert np.isclose(q.mean(), lines["quantity"].describe()["mean"])

2.5546875
2.5546875

✔

الخطوة 5 — اكتب ملخّص الفحص (خمسة أسطر وسؤال).

1. في order_lines.csv يوجد 384 صفًا و7 أعمدة؛ وفي الملف 385 سطرًا، أي الترويسة و384 صف بيانات — الملف كامل.
2. الأعمدة: order_id وline_no وorder_date وregion_code وproduct_id وquantity وunit_price.
3. تمتد التواريخ من 2026-08-01 إلى 2026-10-30؛ و200 طلب مختلف، أي حوالي 1.9 سطر لكل طلب.
4. الأنواع: quantity وline_no أعداد صحيحة، وunit_price عدد عشري (float)؛ وorder_date حُمّل نصًا (str) لا تاريخًا.
5. الكمية من 1 إلى 10؛ وunit_price من 3.60 إلى 42.00؛ ولا قيم مفقودة في أي عمود (.info() تعرض 384 non-null في كل الأعمدة).

السؤال: أقل unit_price هو 3.60، فهل يوجد منتج سعر قائمته بهذا الانخفاض، أم أن 3.60 سعر بعد خصم؟ أحتاج إلى جدول المنتجات لأتحقق.

عندما يفشل أحد الفحوص (مثلًا لا يطابق عدد الصفوف الملف)، توقّف. لا تنتقل إلى الحسابات. افتح الملف في محرر نصوص، وانظر إلى نهايته، ودوّن ما تراه.

---

6. شرح بصري

            column labels  →   order_id  line_no  order_date  region_code  product_id  quantity  unit_price
  index  ┌─────────────────┬──────────┬────────┬────────────┬────────────┬───────────┬─────────┬───────────┐
  (row   │ 0               │ O10056   │   1    │ 2026-08-01 │    HAR     │    P07    │    1    │   35.00   │
  labels)│ 1               │ O10056   │   2    │ 2026-08-01 │    HAR     │    P08    │    2    │    6.00   │
    ↓    │ …               │   …      │   …    │     …      │     …      │     …     │    …    │     …     │
         └─────────────────┴──────────┴────────┴────────────┴────────────┴───────────┴─────────┴───────────┘
                              str       int64       str          str          str       int64     float64
                                                    ▲ looks like a date, loaded as text
  one column  = a Series (values + the same index)          .shape = (384, 7)

وصف الرسم: مخطط جدولي لـ DataFrame أسطر الطلبات. في الأعلى تسميات الأعمدة السبعة: order_id وline_no وorder_date وregion_code وproduct_id وquantity وunit_price. وعلى الجانب الأيسر الفهرس، أي تسميات الصفوف 0 و1 وهكذا. ويظهر صفّان كمثال. وتحت كل عمود نوع بياناته: str لـ order_id، وint64 لـ line_no، وstr لـ order_date، وstr لـ region_code، وstr لـ product_id، وint64 لـ quantity، وfloat64 لـ unit_price. وسهم يشير إلى order_date: يبدو تاريخًا لكنه حُمّل نصًا. وملاحظة تقول إن العمود الواحد Series، مكوّن من قيم مع الفهرس نفسه، وإن الشكل 384 صفًا في 7 أعمدة. (نصوص الرسم: column labels = تسميات الأعمدة؛ index (row labels) = الفهرس (تسميات الصفوف)؛ looks like a date, loaded as text = يبدو تاريخًا، وحُمّل نصًا؛ one column = a Series (values + the same index) = العمود الواحد Series (قيم + الفهرس نفسه).)

---

7. تدريب موجَّه

افتح L25_starter.ipynb في practice_files. ستفحص u07_data/orders.csv: صف واحد لكل طلب. طبّق قائمة الفحص من الجزء E. اكتب ما تتوقعه قبل كل خطوة.

1. الحجم المتوقع. عُدّ أسطر orders.csv من دون pandas، كما في المثال المحلول.
2. التحميل. اقرأ الملف في DataFrame اسمه orders. وتحقّق بـ assert من أن عدد الصفوف يساوي عدد الأسطر ناقص 1.
3. الأسماء ونظرة. اطبع تسميات الأعمدة، وأول 5 صفوف، وآخر 3 صفوف.
4. الأنواع. شغّل orders.info(). ما الأعمدة النصية؟ وما الأعمدة الرقمية؟ هل نوع أي عمود مفاجئ؟
5. الأرقام. اطبع orders.describe(). وأعد حساب متوسط order_value باستخدام NumPy وتحقّق منه بـ np.isclose.
6. الفئات. اطبع value_counts() للعمود status وللعمود region_code. وتحقّق من أن مجموع كل مجموعة من الأعداد يساوي عدد الصفوف.
7. الملخّص. اكتب خمسة أسطر وسؤالًا واحدًا، كما في الخطوة 5 من المثال المحلول.
8. شغّل Restart Kernel and Run All Cells… وتأكد من أن كل الخلايا تعمل بلا أخطاء.

---

8. تدريب مستقل

حمّل u07_data/products.csv (الجدول الرئيسي للمنتجات). واكتب ملخّص فحص في خلية Markdown داخل دفترك الخاص L25_products.ipynb:

1. الصفوف والأعمدة، مع التحقق منها مقابل عدد أسطر الملف.
2. أسماء الأعمدة.
3. أنواع البيانات، مع تعليق على أي عمود يبدو نوعه غير مناسب لمعناه. لا تُصلحه: إصلاح الأنواع موضوعه U09.
4. أي عمود هو على الأرجح المفتاح (key)، أي العمود الذي يميّز كل صف تمييزًا فريدًا؟ قدّم دليلك: قارن nunique() بعدد الصفوف.
5. ملاحظة واحدة من الإحصاءات الملخِّصة (مثل مدى unit_cost)، وملاحظة واحدة عن الفئات (value_counts() لعمود نصي).
6. سؤال واحد يثيره الفحص.

---

9. اختبار الفهم (Knowledge Check) — الفحص

Q1. تُرجع df.shape القيمة (200, 7). ماذا يعني ذلك؟

• A) 200 عمود و7 صفوف
• B) 200 صف و7 أعمدة
• C) 200 خلية في المجموع
• D) 7 قيم مفقودة

Q2. ملف CSV فيه 51 سطرًا، منها سطر الترويسة. بعد pd.read_csv بالإعدادات الافتراضية، كم صفًا يجب أن يكون في الـ DataFrame؟

• A) 50
• B) 51
• C) 52
• D) لا يمكن معرفة ذلك

Q3. أي عبارة عن الـ Series صحيحة؟

• A) هي جدول ثنائي البعد
• B) هي مصفوفة أحادية البعد من القيم، معها فهرس من التسميات
• C) لا يمكنها أن تحمل إلا النصوص
• D) ليس لها نوع بيانات

Q4. في pandas 3، أي نوع بيانات يظهر عادةً لعمود نصي حمّلته read_csv؟

• A) int64
• B) float64
• C) str
• D) datetime64

Q5. الدالة orders.describe() لا تعرض العمود status. لماذا؟

• A) العمود فارغ
• B) describe() تلخّص الأعمدة الرقمية افتراضيًا، وstatus نص
• C) اسم العمود مكتوب خطأ
• D) describe() لا تعرض إلا أول خمسة أعمدة

Q6. يقول دفترك FileNotFoundError: ... 'u07_data/orders.csv'، مع أن الملف موجود على جهازك. ما السبب الأرجح؟

• A) الملف كبير جدًا
• B) pandas لا تستطيع قراءة ملفات CSV
• C) مجلد العمل ليس المجلد الذي يحتوي على u07_data
• D) يجب فتح الملف في Excel أولًا

Q7. أي كود يعطي عدد أرقام الطلبات المختلفة؟

• A) lines["order_id"].count()
• B) len(lines)
• C) lines["order_id"].nunique()
• D) lines.shape[1]

Q8. عمود اسمه boxes_counted حُمّل بنوع البيانات str بدلًا من رقم. ما أفضل إجراء أول في L25؟

• A) احذف العمود
• B) دوّنه في ملخّص الفحص وانظر إلى value_counts() الخاص به لترى أي القيم ليست أرقامًا
• C) غيّر كل القيم إلى 0
• D) تجاهله، لأن describe() ستتعامل معه

---

10. الذكاء الاصطناعي في التطبيق (اختياري، وفي حدود)

الهدف: أن تطلب من مساعد ذكاء اصطناعي أن يشرح ناتج .info()، ثم تتحقق من كل رقم يعطيك إياه مقابل فحصك أنت.

1. الصق فقط ناتج .info() من تدريبك الموجَّه (orders.info()). إنه مصطنع. واسأل:

"I am learning pandas 3. Here is the .info() output of a synthetic order table. Explain each part. How many rows and columns are there? Are there missing values? Which columns are text? Is any type surprising for what the column name suggests?"

وبالعربية: "أتعلّم pandas 3. هذا ناتج .info() لجدول طلبات مصطنع. اشرح كل جزء منه. كم عدد الصفوف والأعمدة؟ هل توجد قيم مفقودة؟ ما الأعمدة النصية؟ هل يوجد نوع مفاجئ بالنسبة إلى ما يوحي به اسم العمود؟"

2. قبل قراءة الإجابة، اكتب أرقامك أنت: الصفوف، والأعمدة، والقيم المفقودة في كل عمود، والأعمدة النصية.
3. قارن، مستعينًا بالقسم 11.

الحدود:

• استخدام الذكاء الاصطناعي اختياري.
• لا تلصق إلا مخرجات مصطنعة مثل هذه. ولا تلصق أبدًا بيانات عملاء حقيقية أو ملفات تصدير حقيقية.
• شروحات الذكاء الاصطناعي المبنية على إصدارات pandas الأقدم كثيرًا ما تقول object للأعمدة النصية؛ أما في pandas 3 فسترى str.

---

11. التحقق من مخرجات الذكاء الاصطناعي

1. الصفوف والأعمدة: يجب أن تساوي أرقام الذكاء الاصطناعي orders.shape. وإذا زاد عدد الصفوف عنده واحدًا على orders.shape[0]، فالأرجح أنه عدّ سطر الترويسة.
2. القيم المفقودة: قارن ادعاءه بعمود Non-Null Count وبناتج orders.isna().sum().
3. الأعمدة النصية: تحقّق منها مقابل orders.dtypes.
4. النوع المفاجئ: هل علّق على أي عمود لا يطابق نوعه ما يوحي به اسمه؟ قارن ذلك بإجابتك أنت في الخطوة 4 من التدريب الموجَّه.
5. ارفض التفاصيل المخترعة. إذا ذكر الذكاء الاصطناعي عمودًا أو قيمة غير موجودين في ناتجك، فهو يخمّن. دوّن ما كان خاطئًا.

---

12. منظور الأعمال

ما الذي تسلّمناه فعلًا؟ كل تقرير يبدأ بملف أعدّه شخص آخر: ملف تصدير من نظام المتجر، أو جدول من زميل، أو ملف منزَّل من مورّد. والملفات كثيرًا ما لا تطابق ما جاء في الطلب:

• شهر ناقص، أو يوم مكرَّر؛
• عمود "أرقام" فيه قيمة مثل 1,200 (بفاصلة الآلاف) أو approx 40، فيُحمَّل نصًا؛
• رموز المناطق ليست هي الموجودة في خريطة المبيعات.

ملخّص فحص من خمسة أسطر يستغرق عشر دقائق، لكنه يحمي كل رقم يأتي بعده. ويمنحك كذلك طريقة مهنية لبدء الحديث مع من أرسل الملف: "The file has [number] orders from [first date] to [last date], and [number] are marked cancelled. Should cancelled orders count in revenue?" (أي: "في الملف [عدد] طلب من [أول تاريخ] إلى [آخر تاريخ]، و[عدد] منها مُعلَّم بأنه ملغى. هل تُحتسب الطلبات الملغاة في الإيرادات؟"). وهذا السؤال أثمن من إجمالي واثق مبني على ملف لم يُفحص.

---

13. مهمة عمل — فحص ملف تصدير أكتوبر

نشاط خيالي: Cedar Lane. يكتب لك مديرك:

"Before Monday's review, can you check the October order file the shop system sent? I just need to know it's the full month and whether anything looks odd."

وبالعربية: "قبل مراجعة يوم الاثنين، هل يمكنك فحص ملف طلبات أكتوبر الذي أرسله نظام المتجر؟ أحتاج فقط أن أعرف هل هو الشهر كاملًا، وهل يبدو فيه شيء غريب."

في L25_october_check.ipynb:

1. حمّل u07_data/orders_2026_10.csv. وتحقّق من عدد الصفوف مقابل عدد أسطر الملف.
2. اذكر أول تاريخ طلب وآخره، وعدد الطلبات المختلفة وعدد العملاء المختلفين.
3. اذكر أعداد status وأعداد region_code.
4. أعد حساب إجمالي order_value باستخدام NumPy وقارنه بـ orders_oct["order_value"].sum().
5. اكتب رسالة قصيرة إلى المدير (من أربع إلى ست جمل): ما الذي يحتويه الملف، وهل يبدو كاملًا لشهر أكتوبر، وشيء واحد يبدو غريبًا أو يحتاج إلى قرار، وسؤال واحد.
6. شغّل Restart Kernel and Run All Cells….

---

14. تحدٍّ قصير — ملف لم تره من قبل

نشاط خيالي: Hilltop Honey، وهو نشاط منفصل يبيع برطمانات العسل في كشكين بالسوق. لم تر ملفه من قبل: u07_data/hilltop_stalls.csv.

لا توجد قائمة فحص لهذا التحدي. افحص الملف بالطريقة التي تراها صحيحة، واكتب ملخّص فحص من خمسة أسطر وسؤالًا واحدًا، كما لو كنت تكتبه لزميل.

يجب أن يذكر ملخّصك أي شيء في الأنواع قد يمنعك من حساب إجمالي الإيرادات مباشرة. لا تُصلح أي شيء؛ صِف ما وجدته وما ستسأل عنه.

سلّم عملك قبل أن تتلقى الملاحظات. لا يوجد حل منشور هنا.

---

15. المصادر

مطلوب (حوالي 25 دقيقة):

• 10 minutes to pandas (دليل مستخدم pandas 3.0): القسمان "Object creation" (إنشاء الكائنات) و"Viewing data" (عرض البيانات). https://pandas.pydata.org/docs/user_guide/10min.html (فُتح في 16 أكتوبر 2026.)

موصى به:

• Python for Data Analysis, 3rd edition، تأليف Wes McKinney، الفصل 5 "Getting Started with pandas"، القسمان 5.1 (Series وDataFrame) و5.3 (التلخيص وحساب الإحصاءات الوصفية). متاح للقراءة مجانًا على الإنترنت: https://wesmckinney.com/book/pandas-basics.html. كُتب الكتاب لإصدار أقدم من pandas، ولذلك تظهر فيه الأعمدة النصية بالنوع object.

مرجع (ارجع إليه عند الحاجة):

• pandas: IO tools، قسم read_csv. لا تحتاج هذا الأسبوع إلا إلى الإعدادات الافتراضية؛ وستستخدم U08 الخيارات. https://pandas.pydata.org/docs/user_guide/io.html

---

16. شخصية تتابعها / قدوة

Wes McKinney (اختارته D.World لـ U07).

• حقائق مُتحقَّق منها (في 16 أكتوبر 2026): يكتب في كتابه Python for Data Analysis، الفصل 1، أنه بدأ بناء pandas في أوائل 2008 أثناء عمله في AQR Capital Management. وتقول صفحة "About" الخاصة بـ pandas إن pandas أصبحت مفتوحة المصدر بنهاية 2009. وهو مؤلف كتاب Python for Data Analysis.
• لماذا توصي به D.World: هو من أنشأ المكتبة التي تستخدمها هذه الوحدة كلها، وكتابه (الفصل 5، الموصى به أعلاه) يشرح Series وDataFrame بأمثلة صغيرة وواضحة.
• ملاحظة صادقة: لا نذكر إلا ما يقوله كتابه وموقع pandas. فـ pandas يطوّرها الآن مجتمع كبير، وأدواره الحالية تتغير مع الوقت.
• جرّب هذا: افتح صفحة "About" الخاصة بـ pandas (https://pandas.pydata.org/about/) واكتب جملة واحدة عن الطريقة التي يُدعم بها المشروع اليوم.

---

17. المصطلحات

المصطلح	معناه في هذا الدرس
Series	مصفوفة أحادية البعد ذات تسميات: قيم مع فهرس
DataFrame (جدول بيانات)	جدول من أعمدة (Series) تشترك في الفهرس نفسه
الفهرس (Index)	تسميات الصفوف في Series أو DataFrame؛ وهي افتراضيًا 0، 1، 2، …
تسمية العمود (Column label)	اسم عمود في DataFrame، مثل "order_value"
نوع البيانات في pandas (pandas dtype)	نوع بيانات العمود، مثل int64 وfloat64 وstr
النوع النصي (String dtype)	النوع الافتراضي للأعمدة النصية في pandas 3، ويظهر بالاسم str (وتعرض الإصدارات الأقدم object)
مسار الملف (File path)	موقع الملف، مكتوبًا على هيئة مجلدات واسم ملف
المسار النسبي (Relative path)	مسار يُقرأ بدءًا من مجلد العمل، مثل u07_data/orders.csv
مجلد العمل (Working directory)	المجلد الذي "يقف فيه" Python حاليًا؛ وفي Jupyter يكون عادةً مجلد الدفتر

للمراجعة: القاموس dict (L15) · المصفوفة والشكل ونوع البيانات array وshape وdtype (L21).

---

18. تأمّل

1. في جملة واحدة لكل منهما، اشرح الفرق بين Series وDataFrame، والفرق بين الفهرس والعمود.
2. أي خطوة من قائمة الفحص كشفت أكثر المعلومات فائدة اليوم؟ ولماذا؟
3. اكتب سؤالًا واحدًا عن بيانات Cedar Lane لا تستطيع الإجابة عنه بعد، لكنك تتوقع أن تجيب عنه بنهاية U07.

---

19. الخطوة التالية

أصبحت الآن قادرًا على تحميل جدول ووصف ما فيه. والتالي:

• L26 — الاختيار والتصفية والترتيب (Selecting, Filtering & Sorting). ستنتقي الصفوف والأعمدة التي تجيب عن سؤال عمل، مثل "كل الطلبات المتأخرة التي تزيد قيمتها على حدّ معيّن"، وتذكر عدد الصفوف في الإجابة.

---

20. أدلة التقييم

الدليل	ما الذي يُظهره	كيف يُقيَّم
قائمة فحص مكتملة (التدريب الموجَّه، orders.csv)	يحمّل ملف CSV، ويتحقق من عدد الصفوف بشكل مستقل، ويقرأ الأنواع والإحصاءات الملخِّصة والفئات	دليل التقييم L25-GP
ملخّص فحص مكتوب (التدريب المستقل، products.csv)	يذكر الحجم والأسماء والأنواع والمفتاح المرجّح مع الدليل، وسؤالًا	دليل التقييم L25-IP
اختبار الفهم (Q1–Q8)	يقرأ الأشكال وأنواع البيانات؛ ويفهم المسارات النسبية؛ ويختار إجراءات أولى سليمة	7 إجابات صحيحة أو أكثر من 8
مهمة العمل والتحدي القصير	يكتب رسالة فحص جاهزة للمدير؛ ويفحص ملفًا لم يره من قبل دون قائمة فحص	دليلا التقييم L25-WT وL25-MC

---

المراجع

1. pandas development team. 10 minutes to pandas (pandas 3.0 user guide). https://pandas.pydata.org/docs/user_guide/10min.html (R04). يدعم: إنشاء Series وDataFrame، وhead وtail وdescribe وto_numpy.
2. pandas development team. IO tools (text, CSV, HDF5, …). https://pandas.pydata.org/docs/user_guide/io.html (P-IO). يدعم: الإعدادات الافتراضية لـ read_csv (الفاصلة بين القيم، والسطر الأول ترويسة).
3. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 5 "Getting Started with pandas". https://wesmckinney.com/book/pandas-basics.html (WM-CH5). يدعم: Series وDataFrame والفهرس والإحصاءات الوصفية.
4. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 1 "Preliminaries", section on pandas. https://wesmckinney.com/book/preliminaries.html (WM-CH1). يدعم حقيقة القدوة الخاصة بعام 2008 وAQR Capital Management.
5. pandas development team. About pandas. https://pandas.pydata.org/about/ (P-ABOUT). يدعم: أصبحت مفتوحة المصدر بنهاية 2009.
6. D.World Stage 2 Detailed Lesson Map v1.0 (صف L25). وثيقة داخلية.

L30 — CSV, Excel & Text Encodings

Learner file. Answers and model solutions are in a separate instructor-only file. Version: U08 Production v1 · Content date: 18 October 2026. Time estimates are planning estimates until learners have tested them. The code in this lesson was run with the course setup from L21 (Python 3.13, pandas 3.0 and NumPy 2.5; openpyxl 3.1.5 where used) on Linux and on Windows 11 on 4 November 2026 (see the Code Test Report).

All businesses are fictional and all data is synthetic.

---

1. Lesson Identity

Field	Value
Title	CSV, Excel & Text Encodings
Stage	Stage 2 — Data Handling
Unit	U08 — Data Acquisition & Data Formats
Lesson ID	L30
Skills	Reading non-default CSV files; reading Excel sheets; encodings; setting types at load time; load checks
Difficulty	Intermediate
Estimated time	About 2 hours (planning estimate, unvalidated)
Assessment level	Application
Prerequisites	L25 (read_csv with defaults, shape, dtypes), L28 (row-count checks), L29 (export to CSV and Excel).

Time breakdown (planning estimate):

• Concept and worked example: 50 min.
• Guided practice: 20 min.
• Independent practice: 25 min.
• Knowledge check: 10 min.
• AI in Practice and Verification: 5 min (optional).
• Workplace task and mini challenge: 10 min.

What you need: your L21 environment and practice_files (L30_starter.ipynb; in u08_data: greenleaf_prices.csv, stock_counts.xlsx, bluepine_catalog.csv, delivery_slots.csv, loyalty_members.csv, hilltop_market_days.txt). Nothing new to install: openpyxl (L21) is what pandas uses to read .xlsx files.

---

2. Learning Objective

By the end of this lesson, you will be able to:

• read CSV and Excel files that break pandas' defaults (delimiter, decimal separator, header row, sheet name, encoding, ID columns);
• diagnose a file's quirk by looking at its raw text before choosing read options;
• confirm that each loaded table matches its source by row count, columns and a known value, and record this in a load log.

D.World approved objective (verbatim): Read CSV and Excel files that break pandas' defaults (delimiter, decimal separator, header row, sheet name, encoding, ID columns) and confirm each loaded table matches its source by row count, columns and known values.

---

3. Why It Matters

In U07 every file was a "clean" CSV: commas between columns, a dot for decimals, one header row, plain English text. Real files are rarely like that. Cedar Lane (fictional) buys from suppliers in several countries, and each one sends price lists in its own regional format:

• one uses semicolons between columns because its decimal separator is a comma (3,45);
• one exports from an old system in a non-UTF-8 encoding, so Café arrives as bytes pandas cannot read with its default;
• one puts a title above the header row;
• one sends a spreadsheet where a product code like 004120 has silently become the number 4120.

The dangerous cases are the ones that do not raise an error. A wrong delimiter can give you one wide column; a wrong decimal setting can give you text instead of numbers; a lost leading zero makes a code that no longer matches any other table. The table looks loaded.

Key principle: a file is not loaded until you have checked it against the source — rows, columns, types and one known value — and written down which options you used and why.

---

4. Concept

import io
from pathlib import Path

import pandas as pd

DATA = Path("u08_data")
print(sorted(p.name for p in DATA.glob("*.csv")))

['bluepine_catalog.csv', 'delivery_slots.csv', 'greenleaf_prices.csv', 'loyalty_members.csv']

Part A — Look at the raw text first

read_csv makes guesses. Before you let it guess, read the first lines yourself as plain text. Python can do this without pandas:

with open(DATA / "greenleaf_prices.csv", encoding="utf-8") as f:
    for _ in range(3):
        print(repr(f.readline()))

'sku;product;pack;unit_price;min_order\n'
'000904;Rolled oats;1 kg;0,90;6\n'
'001017;Basmati rice;2 kg;8,19;12\n'

repr() shows the line exactly, including the newline \n. Three things are visible at once: columns are separated by ;, decimals use a comma (0,90), and the codes start with zeros (000904). Each needs a read option.

CSV structure in brief. A CSV file is plain text: one record per line, fields separated by a delimiter, usually a header row with the column names first. When a field itself contains the delimiter, it is wrapped in double quotes (quoting), for example "Oats, rolled". The informational standard RFC 4180 describes this common format; many real files deviate from it, which is why you look first.

quoted = 'sku,product,price\n000904,"Oats, rolled",0.90\n'
print(pd.read_csv(io.StringIO(quoted), dtype={"sku": "str"}))

      sku       product  price
0  000904  Oats, rolled    0.9

io.StringIO turns a string into something that behaves like a file, which is handy for small experiments. The comma inside the quotes stayed in the product name.

Part B — Delimiter, decimal separator and header row

The four options you will use most:

Option	Default	Use when
sep=	","	fields are separated by something else: ";", "\t" (tab), "|"
decimal=	"."	numbers use a comma as the decimal separator (3,45)
header= / skiprows=	header=0	there are title lines above the column names
usecols=	all columns	you only need some columns

See what happens with the defaults on a small semicolon file:

semi = "code;price\n001;3,45\n002;10,10\n"
wrong = pd.read_csv(io.StringIO(semi))
print(wrong.shape, list(wrong.columns))
right = pd.read_csv(io.StringIO(semi), sep=";", decimal=",", dtype={"code": "str"})
print(right.shape, right.dtypes.astype(str).to_dict())
print(right["price"].sum())

(2, 1) ['code;price']
(2, 2) {'code': 'str', 'price': 'float64'}
13.55

With the defaults there is no error: you get one column called code;price. With sep=";" and decimal="," the price becomes a real number you can add up.

Title lines. If a file starts with lines that are not data, skiprows=n skips the first n lines, then the next line is the header:

titled = "Supplier export\nWeek 42\ncode,qty\nA1,4\nA2,7\n"
print(pd.read_csv(io.StringIO(titled), skiprows=2))

  code  qty
0   A1    4
1   A2    7

Without skiprows, pandas would take Supplier export as the column name and then fail or misread the rest.

Part C — Types at load time: keep IDs as text

pandas infers a type for each column. A column that looks numeric becomes a number, even when it is an identifier (an ID, a product code, a postcode). Numbers have no leading zeros, so 000904 becomes 904, and it will no longer match 000904 in another table.

ids = "member_id,points\n000731,120\n004410,85\n"
print(pd.read_csv(io.StringIO(ids))["member_id"].tolist())
print(pd.read_csv(io.StringIO(ids), dtype={"member_id": "str"})["member_id"].tolist())

[731, 4410]
['000731', '004410']

Rule: when you know a column's meaning before loading, set its type with dtype= (a dictionary {column: type}). Use "str" for codes and IDs, even if they contain only digits. A useful test: would you ever add or average this column? If not, it is not a number.

Dates can also be typed at load with parse_dates=["column"] when the format is clean and you know the column is a date. In U08 we keep dates as text unless a task asks otherwise; repairing types that were loaded wrongly is a U09 job (L37).

Part D — Encodings

A text file is stored as bytes. An encoding is the rule that turns characters into bytes and back. UTF-8 is the modern standard: it can store every character in every language, and it is pandas' default. Older systems often write a regional encoding such as Windows-1252 (cp1252 in Python), where é is a single byte 0xE9. In UTF-8 that byte on its own is invalid, so reading the file as UTF-8 fails:

raw = "code,name\nA1,Café crème\n".encode("cp1252")
print(raw)
try:
    pd.read_csv(io.BytesIO(raw))
except UnicodeDecodeError as e:
    print("UnicodeDecodeError:", e)

b'code,name\nA1,Caf\xe9 cr\xe8me\n'
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 16: invalid continuation byte

The fix is not to delete the strange characters. It is to tell pandas the file's real encoding:

print(pd.read_csv(io.BytesIO(raw), encoding="cp1252"))

  code        name
0   A1  Café crème

Careful: some wrong encodings never raise an error. latin-1 can decode any byte, so it always "works", but some characters come out wrong. Check the text, not just the absence of an error:

raw2 = "code,name\nA2,Œuf noodles\n".encode("cp1252")
print(repr(pd.read_csv(io.BytesIO(raw2), encoding="latin-1").loc[0, "name"]))
print(repr(pd.read_csv(io.BytesIO(raw2), encoding="cp1252").loc[0, "name"]))

'\x8cuf noodles'
'Œuf noodles'

With latin-1, the Œ became an invisible control character. How do you choose? Ask the sender, read the system's export settings, and check a known value with characters such as é, ñ, ü or Œ. If you see é instead of é, a UTF-8 file was read with a regional encoding: the opposite mistake.

Part E — Excel workbooks and sheets

An Excel workbook (.xlsx) can hold several sheets. pd.read_excel reads one sheet; sheet_name= chooses which, and sheet_name=None reads all of them into a dictionary {sheet name: DataFrame}:

book = pd.read_excel(DATA / "stock_counts.xlsx", sheet_name=None)
print(list(book))
print({name: df.shape for name, df in book.items()})

['Warehouse', 'Stores']
{'Warehouse': (16, 3), 'Stores': (26, 3)}

read_excel accepts the same header=, skiprows=, usecols= and dtype= options as read_csv. Spreadsheets change data before you ever load it. When someone types 004120 into a cell formatted as a number, the spreadsheet stores the number 4120. When someone types 3/4, it may become a date. pandas cannot recover what the spreadsheet already changed; you can only notice it (and report it) by comparing with the source.

And even when the cell does contain the text "000904", read_excel infers types like read_csv does, so a digits-only code becomes a number unless you set dtype=. You will see both effects in the guided practice.

Part F — Python's csv module, for awareness

Python's built-in csv module reads a file row by row as lists of strings, with no type guessing. It is useful for checking a file independently of pandas, for example counting data rows:

import csv

with open(DATA / "greenleaf_prices.csv", encoding="utf-8", newline="") as f:
    rows = list(csv.reader(f, delimiter=";"))
print(len(rows) - 1, rows[0])

24 ['sku', 'product', 'pack', 'unit_price', 'min_order']

You will use pandas for analysis; csv gives you a second, simpler opinion when a load looks suspicious.

Part G — The load check and the load log

Every load ends with the same four checks:

1. Rows: the row count equals the number of data lines in the source.
2. Columns: the expected names, and the number of columns, are right.
3. Types: IDs are text; numeric columns are numeric.
4. A known value: one cell you can read in the raw text appears correctly in the table.

Then you write a short load log entry: file, quirk(s) seen, options used, checks performed and their result. The log is how a colleague can rerun your load, and how you will explain it in the U08 exit check.

---

5. Worked Example

Cedar Lane (fictional), synthetic data. The supplier Greenleaf sends greenleaf_prices.csv. Load it for analysis and prove the load is right.

Step 1 — Read the raw text. (Part A showed it: ; delimiter, decimal comma, codes with leading zeros, UTF-8 text with no special characters.)

Step 2 — Count data lines without pandas.

with open(DATA / "greenleaf_prices.csv", encoding="utf-8") as f:
    n_lines = sum(1 for _ in f)
print(n_lines, "lines ->", n_lines - 1, "data rows expected")

25 lines -> 24 data rows expected

Step 3 — Load with options chosen from the raw text.

greenleaf = pd.read_csv(DATA / "greenleaf_prices.csv", sep=";", decimal=",", dtype={"sku": "str"})
print(greenleaf.shape)
print(greenleaf.dtypes.astype(str).to_dict())

(24, 5)
{'sku': 'str', 'product': 'str', 'pack': 'str', 'unit_price': 'float64', 'min_order': 'int64'}

Step 4 — Checks.

assert len(greenleaf) == n_lines - 1
assert list(greenleaf.columns) == ["sku", "product", "pack", "unit_price", "min_order"]
assert greenleaf["sku"].str.len().eq(6).all()
assert pd.api.types.is_float_dtype(greenleaf["unit_price"])
coffee = greenleaf.loc[greenleaf["product"] == "Ground coffee", ["sku", "unit_price"]]
print(coffee.to_string(index=False))

   sku  unit_price
003391       12.85

The raw line for ground coffee reads …;Ground coffee;500 g;12,85;…, and the table shows 12.85 as a number. ✔

Step 5 — The load log entry.

File: u08_data/greenleaf_prices.csv (Greenleaf, fictional supplier). Quirks seen in raw text: ; delimiter; decimal comma; 6-digit SKUs with leading zeros. Options: sep=";", decimal=",", dtype={"sku": "str"}; encoding UTF-8 (default, no special characters seen). Checks: rows 24 = 25 lines − 1 ✔; 5 expected columns ✔; every SKU has 6 characters ✔; unit_price is float ✔; known value Ground coffee = 12.85 ✔. Not fixed / questions: none.

---

6. Visual Explanation

 RAW TEXT (what is in the file)            OPTION                 LOADED TABLE (what pandas made)
 ─────────────────────────────────         ──────────────────     ─────────────────────────────────
 sku;product;pack;unit_price               sep=";"           →     5 columns, not 1
 000904;Rolled oats;1 kg;0,90              decimal=","       →     unit_price = 0.90 (float64)
 000904                                    dtype={"sku":"str"} →   "000904", not 904
 Caf\xe9 (byte E9)                         encoding="cp1252" →     "Café", not UnicodeDecodeError
 Title line / Week 42 / header             skiprows=2        →     correct column names
 workbook: [Warehouse] [Stores]            sheet_name=...    →     the sheet you meant

 CHECK EVERY LOAD:  rows = data lines │ column names │ ID and number types │ one known value  →  LOAD LOG

Alt text: A three-column diagram. The left column shows raw file contents: a semicolon-separated header, a row with a decimal comma, a product code with leading zeros, a byte that encodes é in Windows-1252, a file with two title lines, and a workbook with two sheets. The middle column shows the read option for each: sep, decimal, dtype as text, encoding cp1252, skiprows 2 and sheet_name. The right column shows the correct result: five columns, a float price, the code kept as text with its zeros, the word Café, correct column names and the intended sheet. A bar at the bottom lists the four checks for every load: rows equal data lines, column names, ID and number types, one known value, leading to a load log.

---

7. Guided Practice

Open L30_starter.ipynb. You will load the two-sheet workbook stock_counts.xlsx.

1. Read all sheets with sheet_name=None. Print the sheet names and each sheet's shape.
2. Look at the first five rows of the Stores sheet as loaded with the defaults. What went wrong with the header? Choose header= or skiprows= to fix it.
3. Load Stores again with your option and dtype={"sku": "str"}. Check: 24 data rows (3 stores × 8 SKUs), columns store_code, sku, on_hand, and every SKU has 6 characters.
4. Load Warehouse, also with dtype={"sku": "str"}. Print the first three SKUs. Compare them with the SKUs in greenleaf_prices.csv.
5. Explain in two sentences what happened to the Warehouse SKUs before pandas read the file, and why dtype= cannot undo it.
6. Write a load log entry for each sheet (file and sheet, quirks, options, checks, not fixed / questions). Do not repair the Warehouse codes: report them.
7. Run Restart Kernel and Run All Cells….

---

8. Independent Practice

Three supplier and operations files. In u08_data there are three more files: bluepine_catalog.csv, delivery_slots.csv and loyalty_members.csv. Each one breaks a pandas default in one way.

For each file:

1. Look at the raw text (or the raw bytes) and diagnose the quirk yourself.
2. Choose read options and load the file.
3. Run the four load checks from Part G.
4. Write a load log entry: quirk, option used, checks performed and their results, anything not fixed.

Work in L30_starter.ipynb (Part 2). Do not change the files.

---

9. Knowledge Check — Reading Files Correctly

Q1. pd.read_csv("prices.csv") returns a table with shape (40, 1) and one column named sku;name;price. What is the most likely cause?

• A) The header row is missing
• B) The file has 40 empty columns
• C) The file uses ; as the delimiter
• D) The file is encoded in UTF-8

Q2. A price column loads with dtype str, and its values look like 12,50. Which option fixes this at load time?

• A) encoding="utf-8"
• B) sep=","
• C) skiprows=1
• D) decimal=","

Q3. Why should a product code such as 004120 be loaded with dtype={"code": "str"}?

• A) read_csv refuses numeric codes
• B) As a number it loses its leading zeros and no longer matches other tables
• C) pandas cannot sort numbers
• D) Text columns use less memory

Q4. Reading a file gives UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9. What is the best next step?

• A) Use errors="ignore" so the error disappears
• B) Find out the file's real encoding and pass it with encoding=
• C) Convert the file to Excel
• D) Delete the rows that contain unusual characters

Q5. You read a file with encoding="latin-1". There was no error. What can you conclude?

• A) The file has no special characters
• B) The file is definitely Latin-1
• C) The file is UTF-8
• D) Nothing yet: Latin-1 decodes any byte, so you must check known values with special characters

Q6. A workbook has the sheets About, Jan and Feb. What does pd.read_excel(path) read?

• A) Only the last sheet
• B) All three sheets
• C) Only the first sheet, About
• D) It raises an error because there are several sheets

Q7. Which set of checks confirms that a load matches its source?

• A) Row count vs data lines, column names, ID and numeric types, and one known value
• B) The table has no missing values
• C) The code ran without an error
• D) head() looks reasonable

Q8. A spreadsheet column of product codes shows 4120 where the supplier's list says 004120. In U08, what do you do?

• A) Load with dtype="str" and assume it is fixed
• B) Add the zeros back with code and say nothing
• C) Record it in the load log as a problem introduced before loading and report it; repairing values is a U09 decision
• D) Delete the column

---

10. AI in Practice (optional, bounded)

Goal: ask an AI assistant for read_csv options from a tiny synthetic sample, then verify them yourself.

1. Copy five lines of raw text from greenleaf_prices.csv (synthetic data, safe to share) and ask:

"Here are the first five lines of a CSV file. Which pandas 3 read_csv options should I use so that every column has the right type? Explain each option."

2. Before running its answer, write down the options you would choose.
3. Run its suggestion and follow Section 11.

Boundaries:

• AI is optional.
• Paste only synthetic samples such as this one. Never paste a real supplier's or customer's file.
• Five lines cannot show everything (for example an encoding problem in line 200). The checks on the whole file are your job.

---

11. AI Verification

1. Types: print dtypes. Is sku text and unit_price a float?
2. Leading zeros: does every SKU still have 6 characters?
3. Known value: does Ground coffee show 12.85?
4. Rows: does the row count equal the number of data lines?
5. Write down one option the AI got right and one it missed or explained wrongly.

---

12. Business Lens

Regional formats are normal, not errors. In many countries the comma is the decimal separator, so spreadsheets in those regions export CSV files with semicolons. Suppliers are not doing anything wrong; they are using their local settings. A buyer who loads price lists without checking can produce a cost report that is out by a factor of 100 (3,45 read as 345) or silently missing a supplier's codes.

A load log turns "I opened the file" into something a manager can trust: which options were used, which checks passed, and what was found but not changed. When a supplier changes its export settings next quarter, the log shows exactly which check will fail.

---

13. Workplace Task — The Supplier Onboarding Note

Fictional business: Cedar Lane. Your manager writes:

"We'll get Greenleaf's price list every month. Can you write a short note for whoever loads it next, so they don't have to work it out again?"

Write a supplier onboarding note (80–120 words) for the Greenleaf file. Include:

1. the exact read_csv call;
2. why each option is needed, in plain language;
3. the checks to run after every monthly load, with the expected row count for this month's file;
4. one thing the next person should ask Greenleaf if a check fails.

Then rerun the load in a fresh cell exactly as written in your note, to prove the note works.

---

14. Mini Challenge — Market Days in One Column

Fictional business: Hilltop Honey. A colleague loaded hilltop_market_days.txt:

hm = pd.read_csv(DATA / "hilltop_market_days.txt")
print(hm.shape)

1. Before running it, look at the raw text with repr() and predict the shape.
2. Run it. Explain why there is only one column.
3. Load the file correctly and run the four load checks.
4. Write a one-line load log entry and a one-sentence explanation for the colleague.

Submit before feedback. No solution is provided here.

---

15. Resources

Required (about 30 min):

• pandas: IO tools (text, CSV, HDF5, …), sections "CSV & text files" (options sep, decimal, header, skiprows, usecols, dtype, encoding) and "Excel files" (sheet_name). https://pandas.pydata.org/docs/user_guide/io.html (Opened 18 Oct 2026.)

Recommended:

• Python for Data Analysis, 3rd edition, Ch. 6 "Data Loading, Storage, and File Formats", sections 6.1 (reading and writing text files) and the Excel section. https://wesmckinney.com/book/accessing-data.html
• RFC 4180: Common Format and MIME Type for CSV Files (Informational), Section 2, the rules for delimiters, quoting and header lines. https://www.rfc-editor.org/rfc/rfc4180

Reference (look things up when needed):

• Python docs: csv — CSV file reading and writing. https://docs.python.org/3/library/csv.html
• openpyxl documentation, the library pandas uses for .xlsx files. https://openpyxl.readthedocs.io/en/stable/

---

16. Person to Follow / Role Model

Yakov Shafranovich (mapped by D.World for L30).

• Verified fact: he is the author of RFC 4180 (October 2005), which documents the common CSV format and registers the text/csv MIME type. Source: the RFC itself.
• Why D.World recommends him: RFC 4180 is short and readable. It shows that even "simple" CSV needed written rules, and that many files still do not follow them.
• Honest note: we cite only what the RFC states.
• Try this: read Section 2 of RFC 4180 (seven rules). Which rule does greenleaf_prices.csv not follow?

---

17. Glossary

Term	Meaning in this lesson
Delimiter	The character that separates fields in a line: ,, ;, tab, | (sep= in pandas)
Quoting	Wrapping a field in double quotes so it can contain the delimiter, e.g. "Oats, rolled"
Encoding	The rule that turns text characters into bytes and back (encoding= in pandas)
UTF-8	The standard encoding that can store every character; pandas' default
Header row	The line that holds the column names (header= / skiprows=)
Sheet	One tab of an Excel workbook (sheet_name= in read_excel)
Decimal separator	The character between whole and fractional parts: . or , (decimal=)
Leading zeros	Zeros at the start of a code (000904); lost if the code is stored as a number
Load log	A short record per file: quirks seen, options used, checks and results, anything not fixed

Review: read_csv defaults and dtypes (L25) · export (L29).

---

18. Reflection

1. Which quirk worries you most: one that raises an error, or one that does not? Why?
2. Why is "a known value" a stronger check than "no error"?
3. Which spreadsheet habit at work could damage codes or dates before they reach you?

---

19. Next Step

You can now read flat files in many formats and prove the load is right. Next:

• L31 — JSON & Nested Data. The online store platform does not send flat tables. You will load a JSON export, map its nesting and flatten it into tables with a clear grain.

---

20. Assessment Evidence

Evidence	What it shows	How it is judged
Load logs for three files (Independent Practice)	Diagnoses each quirk from raw text without a hint; chooses options; checks rows, columns, types and a known value	Rubric L30-IP
Guided practice (two-sheet workbook)	Chooses sheets and header rows; keeps codes as text; reports damage done before loading	Rubric L30-GP
Knowledge check (Q1–Q8)	Matches symptoms to read options; treats "no error" with care	7 or more of 8 correct
Workplace task and mini challenge	Writes a rerunnable onboarding note; diagnoses a delimiter problem	Rubrics L30-WT, L30-MC

---

References

1. pandas development team. IO tools (text, CSV, HDF5, …). https://pandas.pydata.org/docs/user_guide/io.html (P-IO). Supports: read_csv options sep, decimal, header, skiprows, usecols, dtype, encoding, parse_dates; read_excel and sheet_name=None returning a dictionary.
2. Shafranovich, Y. RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files (Informational), 2005. https://www.rfc-editor.org/rfc/rfc4180 (RFC4180). Supports: CSV structure, quoting, header line; role-model fact.
3. Python Software Foundation. csv — CSV File Reading and Writing. https://docs.python.org/3/library/csv.html (PY-CSV). Supports: csv.reader, delimiter, opening files with newline="".
4. openpyxl documentation. https://openpyxl.readthedocs.io/en/stable/ (OX-DOC). Supports: .xlsx reading and writing used by pandas.
5. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 6. https://wesmckinney.com/book/accessing-data.html (WM-CH6). Supports: reading text files and Excel files with pandas.
6. D.World Stage 2 Detailed Lesson Map v1.0 (L30 row; unit boundaries: types set at load in L30, repaired after load in L37). Internal.

L30 — ملفات CSV وExcel وترميزات النصوص

ملف المتعلّم. الإجابات والحلول النموذجية في ملف منفصل خاص بالمدرّس فقط. الإصدار: U08 Production v1 · تاريخ المحتوى: 18 أكتوبر 2026. تقديرات الوقت تقديرات للتخطيط إلى أن يختبرها المتعلّمون. الكود في هذا الدرس شُغّل باستخدام تجهيزات الدورة من L21 (Python 3.13 وpandas 3.0 وNumPy 2.5؛ وopenpyxl 3.1.5 حيث استُخدمت) على Linux وعلى Windows 11 في 4 نوفمبر 2026 (راجع تقرير اختبار الكود Code Test Report).

كل الأنشطة التجارية خيالية، وكل البيانات مصطنعة.

---

1. بطاقة الدرس

البند	القيمة
العنوان	ملفات CSV وExcel وترميزات النصوص
المرحلة	المرحلة 2 — التعامل مع البيانات
الوحدة	U08 — الحصول على البيانات وصيغها
رقم الدرس	L30
المهارات	قراءة ملفات CSV غير الافتراضية؛ قراءة أوراق Excel؛ الترميزات (encodings)؛ تحديد الأنواع وقت التحميل؛ فحوص التحميل
المستوى	متوسط
الوقت المتوقع	حوالي ساعتين (تقدير للتخطيط، غير مُتحقَّق منه)
مستوى التقييم	التطبيق
المتطلبات السابقة	L25 (read_csv بالإعدادات الافتراضية، وshape، وdtypes)، وL28 (فحوص عدد الصفوف)، وL29 (التصدير إلى CSV وExcel).

تفصيل الوقت (تقدير للتخطيط):

• المفهوم والمثال المحلول: 50 دقيقة.
• التدريب الموجَّه: 20 دقيقة.
• التدريب المستقل: 25 دقيقة.
• اختبار الفهم: 10 دقائق.
• الذكاء الاصطناعي في التطبيق والتحقق منه: 5 دقائق (اختياري).
• مهمة العمل والتحدي القصير: 10 دقائق.

ما تحتاج إليه: بيئتك من L21 ومجلد practice_files (L30_starter.ipynb؛ وفي u08_data: greenleaf_prices.csv وstock_counts.xlsx وbluepine_catalog.csv وdelivery_slots.csv وloyalty_members.csv وhilltop_market_days.txt). لا شيء جديد للتثبيت: فمكتبة openpyxl (L21) هي التي تستخدمها pandas لقراءة ملفات .xlsx.

---

2. هدف التعلّم

في نهاية هذا الدرس ستكون قادرًا على أن:

• تقرأ ملفات CSV وExcel تخالف الإعدادات الافتراضية لـ pandas (الفاصل بين الحقول، وفاصل الكسور العشرية، وسطر الترويسة، واسم الورقة، والترميز، وأعمدة المعرّفات)؛
• تشخّص خصوصية الملف بالنظر إلى نصه الخام قبل اختيار خيارات القراءة؛
• تتأكد من أن كل جدول محمَّل يطابق مصدره من حيث عدد الصفوف والأعمدة وقيمة معروفة، وتسجّل ذلك في سجل التحميل (load log).

الهدف المعتمد من D.World (بنصّه): Read CSV and Excel files that break pandas' defaults (delimiter, decimal separator, header row, sheet name, encoding, ID columns) and confirm each loaded table matches its source by row count, columns and known values.

---

3. لماذا يهمّك هذا؟

في U07 كان كل ملف CSV "نظيفًا": فواصل بين الأعمدة، ونقطة للكسور العشرية، وسطر ترويسة واحد، ونص إنجليزي بسيط. والملفات الحقيقية نادرًا ما تكون كذلك. تشتري Cedar Lane (نشاط خيالي) من موردين في عدة دول، وكل مورد يرسل قوائم أسعاره بالصيغة الإقليمية الخاصة به:

• أحدهم يستخدم الفاصلة المنقوطة (semicolon) بين الأعمدة لأن فاصل الكسور العشرية عنده فاصلة (3,45)؛
• وآخر يصدّر من نظام قديم بترميز غير UTF-8، فتصل كلمة Café على هيئة بايتات لا تستطيع pandas قراءتها بإعدادها الافتراضي؛
• وثالث يضع عنوانًا فوق سطر الترويسة؛
• ورابع يرسل جدول بيانات تحوّل فيه رمز منتج مثل 004120 بصمت إلى الرقم 4120.

والحالات الخطيرة هي التي لا تُظهر خطأ. فالفاصل الخاطئ قد يعطيك عمودًا واحدًا عريضًا؛ وإعداد الكسور العشرية الخاطئ قد يعطيك نصًا بدلًا من أرقام؛ والصفر البادئ المفقود يصنع رمزًا لم يعد يطابق أي جدول آخر. ومع ذلك يبدو الجدول محمَّلًا.

المبدأ الأساسي: الملف لا يُعتبر محمَّلًا إلا بعد أن تتحقق منه مقابل المصدر — الصفوف والأعمدة والأنواع وقيمة واحدة معروفة — وتدوّن الخيارات التي استخدمتها وسبب استخدامها.

---

4. المفهوم

import io
from pathlib import Path

import pandas as pd

DATA = Path("u08_data")
print(sorted(p.name for p in DATA.glob("*.csv")))

['bluepine_catalog.csv', 'delivery_slots.csv', 'greenleaf_prices.csv', 'loyalty_members.csv']

الجزء A — انظر إلى النص الخام أولًا

الدالة read_csv تخمّن. وقبل أن تتركها تخمّن، اقرأ أول الأسطر بنفسك كنص عادي. ويستطيع Python فعل ذلك من دون pandas:

with open(DATA / "greenleaf_prices.csv", encoding="utf-8") as f:
    for _ in range(3):
        print(repr(f.readline()))

'sku;product;pack;unit_price;min_order\n'
'000904;Rolled oats;1 kg;0,90;6\n'
'001017;Basmati rice;2 kg;8,19;12\n'

الدالة repr() تعرض السطر كما هو بالضبط، بما في ذلك رمز السطر الجديد \n. وتظهر ثلاثة أشياء دفعة واحدة: الأعمدة مفصولة بـ ;، والكسور العشرية تستخدم الفاصلة (0,90)، والرموز تبدأ بأصفار (000904). وكل واحد منها يحتاج إلى خيار قراءة.

بنية ملف CSV باختصار. ملف CSV نص عادي: سجل واحد في كل سطر، وحقول يفصل بينها فاصل (delimiter)، وعادةً يأتي أولًا سطر ترويسة (header row) فيه أسماء الأعمدة. وعندما يحتوي حقل ما على الفاصل نفسه، يُحاط بعلامتي تنصيص مزدوجتين (التنصيص quoting)، مثل "Oats, rolled". ويصف المعيار المعلوماتي RFC 4180 هذه الصيغة الشائعة؛ وكثير من الملفات الحقيقية تخرج عنه، ولهذا تنظر أولًا.

quoted = 'sku,product,price\n000904,"Oats, rolled",0.90\n'
print(pd.read_csv(io.StringIO(quoted), dtype={"sku": "str"}))

      sku       product  price
0  000904  Oats, rolled    0.9

الأداة io.StringIO تحوّل سلسلة نصية إلى شيء يتصرف كأنه ملف، وهذا مفيد للتجارب الصغيرة. وقد بقيت الفاصلة التي بين علامتي التنصيص داخل اسم المنتج.

الجزء B — الفاصل وفاصل الكسور العشرية وسطر الترويسة

الخيارات الأربعة التي ستستخدمها أكثر من غيرها:

الخيار	الافتراضي	استخدمه عندما
sep=	","	تكون الحقول مفصولة بشيء آخر: ";" أو "\t" (مسافة الجدولة tab) أو "|"
decimal=	"."	تستخدم الأرقام الفاصلة فاصلًا للكسور العشرية (decimal separator) (3,45)
header= / skiprows=	header=0	توجد أسطر عنوان فوق أسماء الأعمدة
usecols=	كل الأعمدة	لا تحتاج إلا إلى بعض الأعمدة

انظر ماذا يحدث مع الإعدادات الافتراضية في ملف صغير بالفاصلة المنقوطة:

semi = "code;price\n001;3,45\n002;10,10\n"
wrong = pd.read_csv(io.StringIO(semi))
print(wrong.shape, list(wrong.columns))
right = pd.read_csv(io.StringIO(semi), sep=";", decimal=",", dtype={"code": "str"})
print(right.shape, right.dtypes.astype(str).to_dict())
print(right["price"].sum())

(2, 1) ['code;price']
(2, 2) {'code': 'str', 'price': 'float64'}
13.55

مع الإعدادات الافتراضية لا يظهر أي خطأ: تحصل على عمود واحد اسمه code;price. ومع sep=";" وdecimal="," يصبح السعر رقمًا حقيقيًا يمكنك جمعه.

أسطر العنوان. إذا بدأ الملف بأسطر ليست بيانات، فإن skiprows=n يتخطّى أول n سطر، ثم يكون السطر التالي هو الترويسة:

titled = "Supplier export\nWeek 42\ncode,qty\nA1,4\nA2,7\n"
print(pd.read_csv(io.StringIO(titled), skiprows=2))

  code  qty
0   A1    4
1   A2    7

من دون skiprows كانت pandas ستعتبر Supplier export اسم العمود، ثم تفشل أو تسيء قراءة الباقي.

الجزء C — الأنواع وقت التحميل: أبقِ المعرّفات نصوصًا

تستنتج pandas نوعًا لكل عمود. والعمود الذي يبدو رقميًا يصبح رقمًا، حتى لو كان معرّفًا (identifier) (رقم تعريف، أو رمز منتج، أو رمز بريدي). والأرقام ليس فيها أصفار بادئة (leading zeros)، فيصبح 000904 هو 904، ولن يطابق بعد ذلك 000904 في جدول آخر.

ids = "member_id,points\n000731,120\n004410,85\n"
print(pd.read_csv(io.StringIO(ids))["member_id"].tolist())
print(pd.read_csv(io.StringIO(ids), dtype={"member_id": "str"})["member_id"].tolist())

[731, 4410]
['000731', '004410']

القاعدة: عندما تعرف معنى العمود قبل التحميل، حدّد نوعه بـ dtype= (قاموس {column: type}، أي {العمود: النوع}). واستخدم "str" للرموز والمعرّفات، حتى لو لم تحتوِ إلا على أرقام. واختبار مفيد: هل ستجمع هذا العمود أو تحسب متوسطه في أي وقت؟ إذا كانت الإجابة لا، فهو ليس رقمًا.

ويمكن أيضًا تحديد نوع التواريخ وقت التحميل بـ parse_dates=["column"] عندما تكون الصيغة نظيفة وتعرف أن العمود تاريخ. وفي U08 نُبقي التواريخ نصوصًا ما لم تطلب المهمة غير ذلك؛ أما إصلاح الأنواع التي حُمّلت خطأ فهو مهمة U09 (L37).

الجزء D — الترميزات

يُخزَّن الملف النصي على هيئة بايتات (bytes). والترميز (encoding) هو القاعدة التي تحوّل الحروف إلى بايتات والعكس. وUTF-8 هو المعيار الحديث: يستطيع تخزين كل حرف في كل لغة، وهو الإعداد الافتراضي في pandas. أما الأنظمة الأقدم فكثيرًا ما تكتب بترميز إقليمي مثل Windows-1252 (cp1252 في Python)، حيث يكون الحرف é بايتًا واحدًا هو 0xE9. وهذا البايت وحده غير صالح في UTF-8، ولذلك تفشل قراءة الملف على أنه UTF-8:

raw = "code,name\nA1,Café crème\n".encode("cp1252")
print(raw)
try:
    pd.read_csv(io.BytesIO(raw))
except UnicodeDecodeError as e:
    print("UnicodeDecodeError:", e)

b'code,name\nA1,Caf\xe9 cr\xe8me\n'
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 16: invalid continuation byte

والحل ليس حذف الحروف الغريبة، بل أن تخبر pandas بالترميز الحقيقي للملف:

print(pd.read_csv(io.BytesIO(raw), encoding="cp1252"))

  code        name
0   A1  Café crème

انتبه: بعض الترميزات الخاطئة لا تُظهر خطأ أبدًا. فالترميز latin-1 يستطيع فكّ أي بايت، ولذلك "ينجح" دائمًا، لكن بعض الحروف تخرج خاطئة. افحص النص، لا مجرد غياب الخطأ:

raw2 = "code,name\nA2,Œuf noodles\n".encode("cp1252")
print(repr(pd.read_csv(io.BytesIO(raw2), encoding="latin-1").loc[0, "name"]))
print(repr(pd.read_csv(io.BytesIO(raw2), encoding="cp1252").loc[0, "name"]))

'\x8cuf noodles'
'Œuf noodles'

مع latin-1 أصبح الحرف Œ حرف تحكّم غير مرئي. فكيف تختار؟ اسأل المرسِل، واقرأ إعدادات التصدير في النظام، وافحص قيمة معروفة فيها حروف مثل é أو ñ أو ü أو Œ. وإذا رأيت é بدلًا من é، فهذا ملف UTF-8 قُرئ بترميز إقليمي: أي الخطأ المعاكس.

الجزء E — مصنّفات Excel وأوراقها

مصنّف Excel (.xlsx) قد يضم عدة أوراق (sheets). والدالة pd.read_excel تقرأ ورقة واحدة؛ والخيار sheet_name= يحدد أيّها، وsheet_name=None يقرأ كل الأوراق في قاموس {sheet name: DataFrame}، أي {اسم الورقة: جدول البيانات}:

book = pd.read_excel(DATA / "stock_counts.xlsx", sheet_name=None)
print(list(book))
print({name: df.shape for name, df in book.items()})

['Warehouse', 'Stores']
{'Warehouse': (16, 3), 'Stores': (26, 3)}

تقبل read_excel الخيارات نفسها header= وskiprows= وusecols= وdtype= التي تقبلها read_csv. وجداول البيانات تغيّر البيانات قبل أن تحمّلها أنت أصلًا. فعندما يكتب أحدهم 004120 في خلية منسّقة كرقم، يخزّن البرنامج الرقم 4120. وعندما يكتب 3/4 قد تتحول إلى تاريخ. ولا تستطيع pandas استعادة ما غيّره جدول البيانات بالفعل؛ كل ما تستطيعه أن تلاحظ ذلك (وتبلّغ عنه) بالمقارنة مع المصدر.

وحتى عندما تحتوي الخلية فعلًا على النص "000904"، فإن read_excel تستنتج الأنواع كما تفعل read_csv، فيتحوّل الرمز المكوّن من أرقام فقط إلى رقم ما لم تحدد dtype=. وسترى الأثرين كليهما في التدريب الموجَّه.

الجزء F — وحدة csv في Python، للاطلاع

الوحدة المدمجة csv في Python تقرأ الملف صفًا صفًا على هيئة قوائم من النصوص، من دون أي تخمين للأنواع. وهي مفيدة للتحقق من الملف بشكل مستقل عن pandas، مثل عدّ صفوف البيانات:

import csv

with open(DATA / "greenleaf_prices.csv", encoding="utf-8", newline="") as f:
    rows = list(csv.reader(f, delimiter=";"))
print(len(rows) - 1, rows[0])

24 ['sku', 'product', 'pack', 'unit_price', 'min_order']

ستستخدم pandas للتحليل؛ أما csv فتعطيك رأيًا ثانيًا أبسط عندما يبدو التحميل مريبًا.

الجزء G — فحص التحميل وسجل التحميل

كل تحميل ينتهي بالفحوص الأربعة نفسها:

1. الصفوف: عدد الصفوف يساوي عدد أسطر البيانات في المصدر.
2. الأعمدة: الأسماء المتوقعة وعدد الأعمدة صحيحان.
3. الأنواع: المعرّفات نصوص؛ والأعمدة الرقمية أرقام.
4. قيمة معروفة: خلية واحدة تستطيع قراءتها في النص الخام تظهر صحيحة في الجدول.

ثم تكتب مدخلًا قصيرًا في سجل التحميل: الملف، والخصوصيات التي رأيتها، والخيارات المستخدمة، والفحوص التي أجريتها ونتيجتها. وبهذا السجل يستطيع زميلك إعادة تشغيل تحميلك، وبه ستشرحه في فحص نهاية U08.

---

5. مثال محلول

Cedar Lane (نشاط خيالي)، بيانات مصطنعة. يرسل المورّد Greenleaf الملف greenleaf_prices.csv. حمّله للتحليل وأثبت أن التحميل صحيح.

الخطوة 1 — اقرأ النص الخام. (عرضه الجزء A: الفاصل ;، وفاصلة للكسور العشرية، ورموز بأصفار بادئة، ونص UTF-8 بلا حروف خاصة.)

الخطوة 2 — عُدّ أسطر البيانات من دون pandas.

with open(DATA / "greenleaf_prices.csv", encoding="utf-8") as f:
    n_lines = sum(1 for _ in f)
print(n_lines, "lines ->", n_lines - 1, "data rows expected")

25 lines -> 24 data rows expected

الخطوة 3 — حمّل بخيارات اخترتها من النص الخام.

greenleaf = pd.read_csv(DATA / "greenleaf_prices.csv", sep=";", decimal=",", dtype={"sku": "str"})
print(greenleaf.shape)
print(greenleaf.dtypes.astype(str).to_dict())

(24, 5)
{'sku': 'str', 'product': 'str', 'pack': 'str', 'unit_price': 'float64', 'min_order': 'int64'}

الخطوة 4 — الفحوص.

assert len(greenleaf) == n_lines - 1
assert list(greenleaf.columns) == ["sku", "product", "pack", "unit_price", "min_order"]
assert greenleaf["sku"].str.len().eq(6).all()
assert pd.api.types.is_float_dtype(greenleaf["unit_price"])
coffee = greenleaf.loc[greenleaf["product"] == "Ground coffee", ["sku", "unit_price"]]
print(coffee.to_string(index=False))

   sku  unit_price
003391       12.85

السطر الخام الخاص بالقهوة المطحونة هو …;Ground coffee;500 g;12,85;…، والجدول يعرض 12.85 رقمًا. ✔

الخطوة 5 — مدخل سجل التحميل.

الملف: u08_data/greenleaf_prices.csv (Greenleaf، مورّد خيالي).
الخصوصيات الظاهرة في النص الخام: الفاصل ;؛ فاصلة للكسور العشرية؛ رموز SKU من 6 أرقام بأصفار بادئة.
الخيارات: sep=";" وdecimal="," وdtype={"sku": "str"}؛ والترميز UTF-8 (الافتراضي، ولم تظهر حروف خاصة).
الفحوص: الصفوف 24 = 25 سطرًا − 1 ✔؛ الأعمدة الخمسة المتوقعة ✔؛ كل رمز SKU من 6 حروف ✔؛ unit_price عدد عشري ✔؛ القيمة المعروفة Ground coffee = 12.85 ✔.
ما لم يُصلَح / أسئلة: لا شيء.

---

6. شرح بصري

 RAW TEXT (what is in the file)            OPTION                 LOADED TABLE (what pandas made)
 ─────────────────────────────────         ──────────────────     ─────────────────────────────────
 sku;product;pack;unit_price               sep=";"           →     5 columns, not 1
 000904;Rolled oats;1 kg;0,90              decimal=","       →     unit_price = 0.90 (float64)
 000904                                    dtype={"sku":"str"} →   "000904", not 904
 Caf\xe9 (byte E9)                         encoding="cp1252" →     "Café", not UnicodeDecodeError
 Title line / Week 42 / header             skiprows=2        →     correct column names
 workbook: [Warehouse] [Stores]            sheet_name=...    →     the sheet you meant

 CHECK EVERY LOAD:  rows = data lines │ column names │ ID and number types │ one known value  →  LOAD LOG

وصف الرسم: مخطط من ثلاثة أعمدة. العمود الأيسر يعرض محتوى الملف الخام: ترويسة مفصولة بالفاصلة المنقوطة، وصفًا فيه فاصلة للكسور العشرية، ورمز منتج بأصفار بادئة، وبايتًا يرمّز الحرف é في Windows-1252، وملفًا فيه سطرا عنوان، ومصنّفًا فيه ورقتان. والعمود الأوسط يعرض خيار القراءة لكل حالة: sep وdecimal وdtype كنص وencoding cp1252 وskiprows 2 وsheet_name. والعمود الأيمن يعرض النتيجة الصحيحة: خمسة أعمدة، وسعرًا عشريًا، والرمز محفوظًا كنص بأصفاره، وكلمة Café، وأسماء أعمدة صحيحة، والورقة المقصودة. وشريط في الأسفل يسرد الفحوص الأربعة لكل تحميل: الصفوف تساوي أسطر البيانات، وأسماء الأعمدة، وأنواع المعرّفات والأرقام، وقيمة واحدة معروفة، وكلها تقود إلى سجل التحميل. (نصوص الرسم: RAW TEXT = النص الخام؛ OPTION = الخيار؛ LOADED TABLE = الجدول المحمَّل؛ CHECK EVERY LOAD = افحص كل تحميل.)

---

7. تدريب موجَّه

افتح L30_starter.ipynb. ستحمّل المصنّف ذا الورقتين stock_counts.xlsx.

1. اقرأ كل الأوراق بـ sheet_name=None. واطبع أسماء الأوراق وشكل كل ورقة.
2. انظر إلى أول خمسة صفوف من ورقة Stores كما حُمّلت بالإعدادات الافتراضية. ما الخطأ الذي حدث في الترويسة؟ اختر header= أو skiprows= لإصلاحه.
3. حمّل Stores مرة أخرى بالخيار الذي اخترته ومع dtype={"sku": "str"}. وتحقّق من: 24 صف بيانات (3 متاجر × 8 رموز SKU)، والأعمدة store_code وsku وon_hand، وأن كل رمز SKU من 6 حروف.
4. حمّل Warehouse، أيضًا مع dtype={"sku": "str"}. واطبع أول ثلاثة رموز SKU. وقارنها برموز SKU في greenleaf_prices.csv.
5. اشرح في جملتين ما الذي حدث لرموز SKU في Warehouse قبل أن تقرأ pandas الملف، ولماذا لا يستطيع dtype= التراجع عنه.
6. اكتب مدخلًا في سجل التحميل لكل ورقة (الملف والورقة، والخصوصيات، والخيارات، والفحوص، وما لم يُصلَح / أسئلة). لا تُصلح رموز Warehouse: بلّغ عنها.
7. شغّل Restart Kernel and Run All Cells….

---

8. تدريب مستقل

ثلاثة ملفات للموردين والعمليات. في u08_data ثلاثة ملفات أخرى: bluepine_catalog.csv وdelivery_slots.csv وloyalty_members.csv. وكل واحد منها يخالف إعدادًا افتراضيًا واحدًا في pandas.

لكل ملف:

1. انظر إلى النص الخام (أو البايتات الخام) وشخّص الخصوصية بنفسك.
2. اختر خيارات القراءة وحمّل الملف.
3. أجرِ فحوص التحميل الأربعة من الجزء G.
4. اكتب مدخلًا في سجل التحميل: الخصوصية، والخيار المستخدم، والفحوص التي أجريتها ونتائجها، وأي شيء لم يُصلَح.

اعمل في L30_starter.ipynb (الجزء 2). ولا تغيّر الملفات.

---

9. اختبار الفهم (Knowledge Check) — قراءة الملفات بشكل صحيح

Q1. تُرجع pd.read_csv("prices.csv") جدولًا شكله (40, 1) فيه عمود واحد اسمه sku;name;price. ما السبب الأرجح؟

• A) سطر الترويسة مفقود
• B) في الملف 40 عمودًا فارغًا
• C) الملف يستخدم ; فاصلًا
• D) الملف مرمَّز بـ UTF-8

Q2. عمود أسعار حُمّل بنوع البيانات str، وقيمه تبدو مثل 12,50. أي خيار يصلح هذا وقت التحميل؟

• A) encoding="utf-8"
• B) sep=","
• C) skiprows=1
• D) decimal=","

Q3. لماذا يجب تحميل رمز منتج مثل 004120 مع dtype={"code": "str"}؟

• A) read_csv ترفض الرموز الرقمية
• B) لأنه كرقم يفقد أصفاره البادئة ولا يعود يطابق الجداول الأخرى
• C) pandas لا تستطيع ترتيب الأرقام
• D) الأعمدة النصية تستهلك ذاكرة أقل

Q4. قراءة ملف تعطي UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9. ما أفضل خطوة تالية؟

• A) استخدم errors="ignore" حتى يختفي الخطأ
• B) اعرف الترميز الحقيقي للملف ومرّره بـ encoding=
• C) حوّل الملف إلى Excel
• D) احذف الصفوف التي تحتوي على حروف غير مألوفة

Q5. قرأت ملفًا بـ encoding="latin-1" ولم يظهر خطأ. ماذا تستنتج؟

• A) الملف ليس فيه حروف خاصة
• B) الملف بالتأكيد Latin-1
• C) الملف UTF-8
• D) لا شيء بعد: Latin-1 يفكّ أي بايت، فيجب أن تفحص قيمًا معروفة فيها حروف خاصة

Q6. مصنّف فيه الأوراق About وJan وFeb. ماذا تقرأ pd.read_excel(path)؟

• A) الورقة الأخيرة فقط
• B) الأوراق الثلاث كلها
• C) الورقة الأولى فقط، About
• D) تُظهر خطأ لأن هناك عدة أوراق

Q7. أي مجموعة فحوص تؤكد أن التحميل يطابق مصدره؟

• A) عدد الصفوف مقابل أسطر البيانات، وأسماء الأعمدة، وأنواع المعرّفات والأرقام، وقيمة واحدة معروفة
• B) لا توجد قيم مفقودة في الجدول
• C) الكود عمل بلا خطأ
• D) ناتج head() يبدو معقولًا

Q8. عمود رموز منتجات في جدول بيانات يعرض 4120 بينما تقول قائمة المورّد 004120. ماذا تفعل في U08؟

• A) تحمّل مع dtype="str" وتفترض أن المشكلة حُلّت
• B) تعيد الأصفار بالكود ولا تقول شيئًا
• C) تسجّلها في سجل التحميل على أنها مشكلة حدثت قبل التحميل وتبلّغ عنها؛ وإصلاح القيم قرار من قرارات U09
• D) تحذف العمود

---

10. الذكاء الاصطناعي في التطبيق (اختياري، وفي حدود)

الهدف: أن تطلب من مساعد ذكاء اصطناعي خيارات read_csv من عينة مصطنعة صغيرة، ثم تتحقق منها بنفسك.

1. انسخ خمسة أسطر من النص الخام لـ greenleaf_prices.csv (بيانات مصطنعة، ومشاركتها آمنة) واسأل:

"Here are the first five lines of a CSV file. Which pandas 3 read_csv options should I use so that every column has the right type? Explain each option."

وبالعربية: "هذه أول خمسة أسطر من ملف CSV. ما خيارات read_csv في pandas 3 التي يجب أن أستخدمها حتى يكون لكل عمود النوع الصحيح؟ اشرح كل خيار."

2. قبل تشغيل إجابته، دوّن الخيارات التي كنت ستختارها أنت.
3. شغّل اقتراحه واتّبع القسم 11.

الحدود:

• استخدام الذكاء الاصطناعي اختياري.
• لا تلصق إلا عينات مصطنعة مثل هذه. ولا تلصق أبدًا ملف مورّد حقيقي أو ملف عميل حقيقي.
• خمسة أسطر لا تُظهر كل شيء (مثل مشكلة ترميز في السطر 200). والفحوص على الملف كله مسؤوليتك أنت.

---

11. التحقق من مخرجات الذكاء الاصطناعي

1. الأنواع: اطبع dtypes. هل sku نص وunit_price عدد عشري؟
2. الأصفار البادئة: هل ما زال كل رمز SKU من 6 حروف؟
3. القيمة المعروفة: هل تعرض Ground coffee القيمة 12.85؟
4. الصفوف: هل يساوي عدد الصفوف عدد أسطر البيانات؟
5. دوّن خيارًا واحدًا أصاب فيه الذكاء الاصطناعي، وخيارًا واحدًا أغفله أو شرحه خطأ.

---

12. منظور الأعمال

الصيغ الإقليمية أمر طبيعي، وليست أخطاء. في كثير من الدول تكون الفاصلة هي فاصل الكسور العشرية، ولذلك تصدّر جداول البيانات في تلك المناطق ملفات CSV بالفاصلة المنقوطة. والموردون لا يرتكبون خطأ؛ إنهم يستخدمون إعداداتهم المحلية. والمشتري الذي يحمّل قوائم الأسعار دون فحص قد يُخرج تقرير تكاليف خاطئًا بمعامل 100 (حين تُقرأ 3,45 على أنها 345)، أو تقريرًا تنقصه رموز أحد الموردين بصمت.

وسجل التحميل يحوّل عبارة "فتحت الملف" إلى شيء يستطيع المدير أن يثق به: ما الخيارات المستخدمة، وما الفحوص التي نجحت، وما الذي وُجد ولم يُغيَّر. وعندما يغيّر المورّد إعدادات التصدير في الربع القادم، يُظهر السجل بالضبط أي فحص سيفشل.

---

13. مهمة عمل — مذكرة تجهيز المورّد

نشاط خيالي: Cedar Lane. يكتب لك مديرك:

"We'll get Greenleaf's price list every month. Can you write a short note for whoever loads it next, so they don't have to work it out again?"

وبالعربية: "ستصلنا قائمة أسعار Greenleaf كل شهر. هل يمكنك كتابة مذكرة قصيرة لمن سيحمّلها بعدك، حتى لا يضطر إلى اكتشاف الطريقة من جديد؟"

اكتب مذكرة تجهيز المورّد (supplier onboarding note) (من 80 إلى 120 كلمة) لملف Greenleaf. وضمّنها:

1. استدعاء read_csv بالضبط؛
2. لماذا يلزم كل خيار، بلغة بسيطة؛
3. الفحوص التي تُجرى بعد كل تحميل شهري، مع عدد الصفوف المتوقع لملف هذا الشهر؛
4. شيئًا واحدًا يجب أن يسأل عنه الشخص التالي Greenleaf إذا فشل أحد الفحوص.

ثم أعد تشغيل التحميل في خلية جديدة كما هو مكتوب في مذكرتك تمامًا، لتثبت أن المذكرة تعمل.

---

14. تحدٍّ قصير — أيام السوق في عمود واحد

نشاط خيالي: Hilltop Honey. حمّل أحد الزملاء الملف hilltop_market_days.txt:

hm = pd.read_csv(DATA / "hilltop_market_days.txt")
print(hm.shape)

1. قبل تشغيله، انظر إلى النص الخام بـ repr() وتوقّع الشكل.
2. شغّله. واشرح لماذا يوجد عمود واحد فقط.
3. حمّل الملف بشكل صحيح وأجرِ فحوص التحميل الأربعة.
4. اكتب مدخلًا من سطر واحد في سجل التحميل، وشرحًا من جملة واحدة للزميل.

سلّم عملك قبل أن تتلقى الملاحظات. لا يوجد حل منشور هنا.

---

15. المصادر

مطلوب (حوالي 30 دقيقة):

• pandas: IO tools (text, CSV, HDF5, …)، القسمان "CSV & text files" (الخيارات sep وdecimal وheader وskiprows وusecols وdtype وencoding) و"Excel files" (sheet_name). https://pandas.pydata.org/docs/user_guide/io.html (فُتح في 18 أكتوبر 2026.)

موصى به:

• Python for Data Analysis, 3rd edition، الفصل 6 "Data Loading, Storage, and File Formats"، القسم 6.1 (قراءة الملفات النصية وكتابتها) وقسم Excel. https://wesmckinney.com/book/accessing-data.html
• RFC 4180: Common Format and MIME Type for CSV Files (Informational)، القسم 2، وفيه قواعد الفواصل والتنصيص وأسطر الترويسة. https://www.rfc-editor.org/rfc/rfc4180

مرجع (ارجع إليه عند الحاجة):

• Python docs: csv — CSV file reading and writing. https://docs.python.org/3/library/csv.html
• openpyxl documentation، المكتبة التي تستخدمها pandas لملفات .xlsx. https://openpyxl.readthedocs.io/en/stable/

---

16. شخصية تتابعها / قدوة

Yakov Shafranovich (اختارته D.World لـ L30).

• حقيقة مُتحقَّق منها: هو مؤلف RFC 4180 (أكتوبر 2005)، الذي يوثّق صيغة CSV الشائعة ويسجّل نوع MIME المسمّى text/csv. المصدر: نص الـ RFC نفسه.
• لماذا توصي به D.World: RFC 4180 قصير وسهل القراءة. ويبيّن أن حتى صيغة CSV "البسيطة" احتاجت إلى قواعد مكتوبة، وأن كثيرًا من الملفات ما زالت لا تتبعها.
• ملاحظة صادقة: لا نذكر إلا ما يقوله الـ RFC.
• جرّب هذا: اقرأ القسم 2 من RFC 4180 (سبع قواعد). أي قاعدة لا يتبعها greenleaf_prices.csv؟

---

17. المصطلحات

المصطلح	معناه في هذا الدرس
الفاصل (Delimiter)	الحرف الذي يفصل الحقول في السطر: , أو ; أو tab أو | (sep= في pandas)
التنصيص (Quoting)	إحاطة حقل بعلامتي تنصيص مزدوجتين حتى يستطيع أن يحتوي على الفاصل، مثل "Oats, rolled"
الترميز (Encoding)	القاعدة التي تحوّل حروف النص إلى بايتات والعكس (encoding= في pandas)
UTF-8	الترميز المعياري القادر على تخزين كل الحروف؛ وهو الإعداد الافتراضي في pandas
سطر الترويسة (Header row)	السطر الذي يحمل أسماء الأعمدة (header= / skiprows=)
الورقة (Sheet)	تبويب واحد في مصنّف Excel (sheet_name= في read_excel)
فاصل الكسور العشرية (Decimal separator)	الحرف الذي يفصل الجزء الصحيح عن الجزء الكسري: . أو , (decimal=)
الأصفار البادئة (Leading zeros)	أصفار في بداية الرمز (000904)؛ وتضيع إذا خُزّن الرمز كرقم
سجل التحميل (Load log)	سجل قصير لكل ملف: الخصوصيات الظاهرة، والخيارات المستخدمة، والفحوص ونتائجها، وما لم يُصلَح

للمراجعة: الإعدادات الافتراضية لـ read_csv وأنواع البيانات (L25) · التصدير (L29).

---

18. تأمّل

1. أي خصوصية تقلقك أكثر: التي تُظهر خطأ، أم التي لا تُظهره؟ ولماذا؟
2. لماذا يُعدّ فحص "قيمة معروفة" أقوى من فحص "لا يوجد خطأ"؟
3. أي عادة في استخدام جداول البيانات في العمل قد تُفسد الرموز أو التواريخ قبل أن تصل إليك؟

---

19. الخطوة التالية

أصبحت الآن قادرًا على قراءة الملفات المسطّحة بصيغ كثيرة وإثبات صحة التحميل. والتالي:

• L31 — JSON والبيانات المتداخلة (JSON & Nested Data). منصة المتجر الإلكتروني لا ترسل جداول مسطّحة. ستحمّل ملف تصدير JSON، وترسم خريطة تداخله، وتسطّحه إلى جداول ذات مستوى تفصيل (grain) واضح.

---

20. أدلة التقييم

الدليل	ما الذي يُظهره	كيف يُقيَّم
سجلات تحميل لثلاثة ملفات (التدريب المستقل)	يشخّص كل خصوصية من النص الخام دون تلميح؛ ويختار الخيارات؛ ويفحص الصفوف والأعمدة والأنواع وقيمة معروفة	دليل التقييم L30-IP
التدريب الموجَّه (مصنّف بورقتين)	يختار الأوراق وأسطر الترويسة؛ ويُبقي الرموز نصوصًا؛ ويبلّغ عن الضرر الذي حدث قبل التحميل	دليل التقييم L30-GP
اختبار الفهم (Q1–Q8)	يربط الأعراض بخيارات القراءة؛ ويتعامل بحذر مع "لا يوجد خطأ"	7 إجابات صحيحة أو أكثر من 8
مهمة العمل والتحدي القصير	يكتب مذكرة تجهيز قابلة لإعادة التشغيل؛ ويشخّص مشكلة في الفاصل	دليلا التقييم L30-WT وL30-MC

---

المراجع

1. pandas development team. IO tools (text, CSV, HDF5, …). https://pandas.pydata.org/docs/user_guide/io.html (P-IO). يدعم: خيارات read_csv وهي sep وdecimal وheader وskiprows وusecols وdtype وencoding وparse_dates؛ وread_excel وsheet_name=None الذي يُرجع قاموسًا.
2. Shafranovich, Y. RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files (Informational), 2005. https://www.rfc-editor.org/rfc/rfc4180 (RFC4180). يدعم: بنية CSV والتنصيص وسطر الترويسة؛ وحقيقة القدوة.
3. Python Software Foundation. csv — CSV File Reading and Writing. https://docs.python.org/3/library/csv.html (PY-CSV). يدعم: csv.reader وdelimiter وفتح الملفات مع newline="".
4. openpyxl documentation. https://openpyxl.readthedocs.io/en/stable/ (OX-DOC). يدعم: قراءة ملفات .xlsx وكتابتها كما تستخدمها pandas.
5. McKinney, W. Python for Data Analysis, 3rd ed., Ch. 6. https://wesmckinney.com/book/accessing-data.html (WM-CH6). يدعم: قراءة الملفات النصية وملفات Excel باستخدام pandas.
6. D.World Stage 2 Detailed Lesson Map v1.0 (صف L30؛ حدود الوحدات: تحديد الأنواع وقت التحميل في L30، وإصلاحها بعد التحميل في L37). وثيقة داخلية.