3Blue1Brown has created an amazing youtube video series entitled “The Essence of Linear Algebra”. The series of videos are highly visual and 3Blue1Brown explains linear algebra concepts from a geometric perspective. I must admit that 3Blue1Brown offers explanations and intuitions on linear algebra that I’d wish were taught in all books and lectures. I certainly found myself wishing that I’d had this resource way back when I was a first year engineering student. The custom animations are programmed in Python by 3Blue1Brown.

The Essence of Calculus (YouTube)

“The Essence of Calculus” is another gem from 3Blue1Brown. Also a youtube video series with custom animations, the series will make you feel like youdiscovered Calculus.

The Matrix Calculus You Need For Deep Learning (Online)

Terence Parr and Jeremy Howard (yes thats the same Jeremy Howard from fast.ai fame) explain Matrix Calculus from first principles. If you have programmed a neural network and implemented back propagation, this resource is the next step in completely understanding the underpinning mathematics.

Mathematics for Machine Learning (Book)

Still a work in progress book but chapters for Analytical Geometry, Linear Algebra and Vector Calculus look intuitive and comprehensive.

That’s all for the moment. Please share any useful resources you find in the comments. All the best in your Math travels….

Link: "https://itnext.io/learning-maths-for-machine-learning-and-deep-learning-part-2-ebbbc918724"

Thứ Tư, 20 tháng 12, 2017

Kinh nghiệm học Toán học.

Toán là môn trong những môn học cực kỳ quan trọng cho tất cả học sinh, sinh viên và kể cả nghiên cứu sinh hay những nhà làm khoa học, kinh tế. Sau khi tìm hiểu thông tin về việc tiếp cận đến môn Toán như thế nào để đạt được kết quả tốt cho việc nghiên cứu trong tương lai, tôi đã tham khảo từ hai nguồn hướng dẫn sau:

by Martin Greenhow at http://people.brunel.ac.uk/~mastmmg/ssguide/sshome.htm
by Paul at Math Tutorials

General tips for Studying Mathematics:

Mathematics is cumulative:

You’ve always got to remember that mathematics courses are cumulative. Almost everything you do in a math class will depend on subjects that you’ve previously learned. This goes beyond just knowing the previous sections in your current class to needing to remember material from previous classes.

Bạn phải luôn nhớ rằng các khóa Toán học là 1 sự tích lũy. Hầu hết những điều bạn làm ở lớp toán sẽ phụ thuộc vào những môn học mà bạn học trước đó. Những môn này ngoại trừ những phần trước trong lớp học hiện tại mà cần phải nhớ tài liệu từ những lớp học trước đó.

Listen during class: In order to get something out of class, you need to listen while in class. Sometimes important ideas will not be written down on the board, but instead just spoken by the instructor.
Watch for the things the instructor emphasizes, even if just in words. The more important that an instructor thinks a topic, the more likely that it will show up on the exam.
Ask questions: If you don't understand something then ask your instructor.
Listen when others ask questions: When other students ask questions, make sure you listen both the questions and the answer. It may be that the student asking the question thought of something that you didn't think of.
Review Notes After Class: After each class, you should review your notes. Note the topics that you found confusing and formulate questions that you can ask your instructor or tutor to help you understand the topic.
Budget Adequate Time For Studying/Homework: It often takes more time studying mathematics to learn the subject than you may require in other classes.

Homework will often take more time than you had originally thought it would. Bài tập về nhà sẽ chiếm nhiều thời gian hơn là bạn nghĩ.

Do homework after each class: At the end of each class budget, some time to look over the homework from that days lecture and attempt to do it. Doing this will allow you time to really work at understanding the concepts covered that day.

Do homework without notes and book: After the first few homework problems, put your notes and book up and try to do the remaining problems without referring to your notes and/or book.
Practice, pratic, practice: Practice as mus as possible. The only way to really learn how to do problems is working lots of them. The more you work, the better prepared you will come in exam time.

Taking notes

Listen in class: Do not just write down what you see on the board. No instructor is going to write down every word they say and sometimes the important ideas won't get written down.
Write down explanatory remarks: Make sure you write down any explanatory remarks the instructor makes.
Note important formulas/concepts: If an instructor emphasizes a particular formula or concept then make note of it.
Note topics you don't understand: If you are having trouble understanding something being presented, note that in the margin and at least write down the keywords. Leave yourself a couple of lines so you can fill in the missing details later once you've gotten help to understand the concept.
Review/Edit your notes: As soon you can after class, go back over your notes. Look for any error and/or omissions. Fill in any information you didn't have time to write down in class.
Review regularly: At regular intervals, sit down and review your notes so that you can learn and retain the information. Remember, that this information will probably be required down the road so it's best to learn it as soon as possible.

down the road: in the future

Thứ Hai, 18 tháng 12, 2017

Prerequisite Knowledge

Background knowledge of programming in robotics:

Udacity survey:
Use the below descriptions for the following questions describing your level of familiarity with programming

Level 0: I have never used this language before.
Level 1: I'm learning the basics like variables, conditionals, arithmetic, basic syntax, and string manipulation.
Level 2: I can read other people's code. I can find and fix simple bugs.
Level 3: I can break down problems and I am prepared to reuse existing code, while implementing new functions/classes of my own and specialized libraries. I can trace the execution of most programs with ease.
Level 4: I can write lean, well structured code to solve complex problems utilizing the full extent of the language's feature set.

What best describes your level of familiarity with C++ and Python?
Which best describes your level of familiarity with calculus, linear algebra, probability and statistics, differential equations, Newtonian mechanics?

Recommended Math course from KhanAcademy:

Tutorial on Math Prof. Paul

Differential Equation

Thứ Năm, 7 tháng 12, 2017

What is the difference between derivative and differential?

Roman Andronov, Senior Software Engineer at Dow Jones and Company (1999-present)

Answered Nov 29

Correct me if I’m wrong but I will take a small liberty of modifying the question slightly in order to make it mathematically meaningful:

what is the difference between a derivative of a function at a point and a differential of a function at a point?

The difference is infinitesimally small.

OK, that was a geeky joke. But seriously.

One way to investigate a difference between two objects is to examine their mathematically precise definitions - if those are available.

We are lucky with the definition of a derivative of a function at a point and not much so with the definition of a differential of a function at a point (see the discussion below).

We shall consider only real-to-real single variable functions.

Analytic Definitions

Derivative of a function at a point

Let a function

y (x)

be defined on some interval

I \in R

and let

x_{0} \in I

. The derivative of

y (x)

x_{0}

, denoted as

y_{x}^{'} (x_{0})

, is, by definition, the following limit:

\begin{matrix} (1) & f_{x}^{'} (x_{0}) = lim_{x \to x_{0}} \frac{y (x) - y (x_{0})}{x - x_{0}} \end{matrix}

regardless of whether the limit is a finite real number or is infinite.

If the limit in (1) is a finite real number then it is said that 1)

y (x)

is differentiable at

x_{0}

or 2)

y (x)

possesses/has a derivative at

x_{0}

or 3) a derivative of

y (x)

x_{0}

exists.

Small nitpick. Whether the function is in a single variable or in multiple variables, to avoid any potential ambiguities, it may be a good idea to make it painfully clear over which variable (symbol) the differentiation is carried out - differentiate function blah over (variable) blah.

The repetitive application of the limit in (1) to the resulting object on the left hand side of the equal sign in (1) produces the derivatives of higher order: the second derivative, the third derivative, … , the

n

-th derivative of a function at a point.

We can granulate the definition of (1) further by distinguishing the direction (or the path) from which

x_{0}

is approached - left or right.

I deduce from first principles (of (1)) the derivatives of a number of elementary functions in, these, Quora, answers.

Differential of a function at a point

Let

y (x)

be differentiable at

x_{0}

. Then the differential of

y (x)

x_{0}

, denoted

d y

d y (x_{0})

, is by definition a function linear in

d x

\begin{matrix} (2) & d y = y_{x}^{'} (x_{0}) d x \end{matrix}

where the

d y

symbol is atomic - it is not “dee times why”. As you develop the theory further and generalize, the differential will be a function in two variables,

x

and

d x

d y = d y (x, d x)

A differential of a function can be used as a linear approximation of its actual change

Δ y

in some neighbourhood of

x_{0}

\begin{matrix} Δ y (x_{0}) = y (x_{0} + Δ x) - y (x_{0}) \end{matrix}

Example: let

y (x) = x^{2}

, then:

\begin{matrix} Δ y = (x_{0} + Δ x)^{2} - x_{0}^{2} = \end{matrix}

\begin{matrix} x_{0}^{2} + 2 x_{0} Δ x + Δ x_{0}^{2} - x_{0}^{2} = \end{matrix}

\begin{matrix} 2 x_{0} Δ x + Δ x^{2} \end{matrix}

x_{0} = 3

and

Δ x = 0.01

then:

\begin{matrix} Δ y (3) = 2 \cdot 3 \cdot 0.01 + {0.01}^{2} = 0.0601 \end{matrix}

While:

\begin{matrix} d y (3) = 2 \cdot 3 \cdot 0.001 = 0.0600 \end{matrix}

We see that the difference, read - estimate/computational error, between the actual

Δ y

and

d y

is just

0.0001

The big idea here is that (we hope/expect that) as

Δ x

becomes smaller and smaller (tends to zero) - so will the error. So if a function

y (x)

does posses a (finite) derivative at

x_{0}

then

Δ y

can be represented as:

\begin{matrix} Δ y = y_{x}^{'} (x_{0}) d x + o (Δ x) = d y + o (Δ x) \end{matrix}

where the little o function is usually interpreted as a computational error:

\begin{matrix} lim_{Δ x \to 0} \frac{o (Δ x)}{Δ x} = 0 \end{matrix}

The concept of a differential scales well in both directions: to differentials of higher order and to differentials of functions in multiple variables.

Geometric Interpretation

Consider a graph

g

y (x)

and let

P

be a point on

g

. Construct a tangent

τ

g

through

P

(assume that it can be done).

Then the geometric interpretation of a derivative of

y (x)

P

is that it is the tangent (as a trigonometric function) of the angle

θ

that

τ

forms with the

x

-axis:

\begin{matrix} y_{x}^{'} (P_{x}) = \tan θ \end{matrix}

Give

P_{x}

a small (say) increase

d x

and this is where things get interesting.

Observe that our horizontal change

d x

is applicable in equal measure to both

g

and

τ

\begin{matrix} | P N | = d x \end{matrix}

However, the corresponding vertical change is not:

The corresponding change for

g

Δ y

\begin{matrix} Δ y = | Q N | \end{matrix}

while the corresponding change for

τ

d y

\begin{matrix} d y = | T N | \end{matrix}

One way to see that is as follows: we have one straight line

τ

falling on two straight parallel lines - the

x

-axis and

P_{y} P

extended. Therefore, by Euclid’s B1.P29:

\begin{matrix} ∠ N P T = θ \end{matrix}

Hence:

\begin{matrix} | N T | = | P N | \cdot \tan θ = y_{x}^{'} (P_{x}) d x = d y \end{matrix}

in accordance with (2).

Geometrically (or intuitively) speaking then, the differential of a function at a point is the change in the tangent straight line at a point caused by the change in the function’s independent variable. For a function in two variables its differential will be the change in the tangent plane and so on.

Discussion

In our chain of definitions we hung all our hats on the concept of a limit - a derivative (of a function at a point) was defined in terms of a limit and the corresponding differential was defined in terms of a derivative.

However, we meekly avoided saying anything about this mysterious magnitude of

d x

Δ x

. Trouble is, in a course of a standard analysis

d x

can’t be defined rigorously by modern standards. All we can say about it is that it is “infinitesimally small” or “arbitrarily small”. If we all agree (and this is just an arbitrary agreement) that we can substitute

y = x

into our definition of the differential (2) then we might even “prove” that:

\begin{matrix} d y = d x = x_{x}^{'} \cdot Δ x = 1 \cdot Δ x = Δ x \end{matrix}

\begin{matrix} d x = Δ x \end{matrix}

and, hence, in a lot of textbooks the symbols

d x

and

Δ x

are used interchangeably (if these are used at all).

Historically, the way things shook out is that even though the ideas of the French mathematician Cauchy (1789–1857) (mostly) prevailed - the

ϵ - δ

-driven definition of a limit is the first class citizen in elementary analysis, the ideas and the notation due to Leibniz still linger on.

In M. Spivak’s “Calculus” on the bottom of the page 131 we find: “This ‘infinitely small’ quantity was denoted by

d x

… Although this point of view is impossible to reconcile with (P1)-(P13) properties of the real numbers, some people find this notion of the derivative congenial”.

In a Differential Geometry course you will be told that

d x

is a one-form, that it has to do with a point and a vector and a tangent space and so on.

We’ve talked enough about tangent and derivative. To demonstrate that the fact that

d x

can’t be rigorously defined in a standard analysis course is not the end of the world, let tangent and derivative make an encore appearance - let us manipulate the infinitesimally small entity to compute the derivative of a tangent (what else?)

To that end we steal a right triangle from the above drawing and relabel it to avoid confusion:

The triangle’s base is a unity, therefore, by definition:

\begin{matrix} \tan θ = \frac{| B C |}{| B A |} = | B C | \end{matrix}

Give

θ

an infinitesimally small increment

d θ

by rotating

A C

about

A

counterclockwise (the drawing below is scaled up to gargantuan proportions for demonstration purposes):

Then, the small right (in the limit) triangle highlighted in red will be similar to

△ A B C

by AAA (prove it). By Euclid’s B6.P4 (the sides about the equal angles in similar triangles are in the same proportion):

\begin{matrix} \frac{d \tan θ}{c \cdot d θ} = \frac{| A C |}{| A B |} = \frac{c}{1} \end{matrix}

\begin{matrix} \frac{d \tan θ}{d θ} = c^{2} \end{matrix}

And by the Pythagorean theorem (or by B1.P47):

\begin{matrix} (3) & \frac{d \tan θ}{d θ} = c^{2} = 1 + \tan^{2} θ \end{matrix}

How much effort, would you say, was it to obtain (3)?

Infinitesimally small …