# Complete Pytorch Tensor Tutorial (Initializing Tensors, Math, Indexing, Reshaping)

### Summary

TLDR本视频深入探讨了PyTorch中张量的基本操作，包括张量的初始化、数学运算、索引、重塑等。通过实例演示了如何创建张量、进行元素级运算、矩阵乘法、广播、索引和高级索引技巧。同时，还介绍了如何使用不同的方法来重塑张量，以及如何进行张量的拼接和转置。这些基础知识为深入学习深度学习打下了坚实的基础。

### Takeaways

- 📘 学习基本的张量操作是深入理解PyTorch和深度学习的基础。
- 🔢 张量的初始化可以通过多种方式，如列表嵌套、特定数值填充等。
- 📈 张量可以设置数据类型（如float32）和设备（如CPU或CUDA）。
- 🔧 张量的形状、数据类型和设备等属性可以通过特定的方法进行查询和设置。
- 🔄 张量支持多种数学和比较操作，如加法、减法、除法、指数运算等。
- 🔍 张量索引允许访问和操作张量的特定元素或子张量。
- 🔧 张量可以通过reshape和view方法进行形状变换。
- 🔄 张量可以通过广播机制进行自动维度扩展以执行元素级操作。
- 🔍 张量提供了高级索引功能，如根据条件选择元素、使用高级索引器等。
- 📊 张量操作包括矩阵乘法、矩阵指数运算、元素级乘法等。
- 🔧 张量可以通过特定的函数进行操作，如torch.where、torch.unique、torch.numel等。
- 🔄 张量可以通过concatenate、permute等方法进行维度操作和轴交换。

### Q & A

### 如何初始化一个PyTorch张量？

-可以通过多种方式初始化张量，例如使用列表创建、使用torch.empty创建未初始化的数据、使用torch.zeros创建全零张量、使用torch.rand创建均匀分布的随机数张量等。

### PyTorch张量的数据类型和设备如何设置？

-可以通过设置dtype属性来指定数据类型，如torch.float32。设备可以通过设置device属性来指定，如使用CUDA或CPU。

### 张量的基本属性有哪些？

-张量的基本属性包括其设备（device）、数据类型（dtype）、形状（shape）以及是否需要梯度（requires_grad）。

### 如何在PyTorch中进行张量数学运算？

-PyTorch提供了丰富的张量运算方法，如加法（torch.add）、减法、除法（torch.true_divide）、元素乘法（torch.mul）、矩阵乘法（torch.matmul）等。

### 什么是张量索引？

-张量索引允许我们访问和操作张量的特定元素或子张量。可以通过指定索引、切片、布尔索引等方式进行。

### 如何使用广播（broadcasting）进行张量操作？

-广播允许在形状不完全相同的张量之间进行操作。PyTorch会自动扩展较小的张量以匹配较大张量的形状，然后进行相应的运算。

### 张量重塑（reshaping）是什么意思？

-张量重塑是指改变张量的形状而不改变其数据。可以使用view或reshape方法来实现，其中view要求张量在内存中是连续存储的。

### 如何使用torch.cat进行张量拼接？

-torch.cat用于将多个张量沿着指定的维度拼接。需要将张量作为元组传递，并指定拼接的维度。

### 张量转置（transpose）是如何实现的？

-张量转置可以通过permute方法实现，指定新的维度顺序。对于二维张量，可以直接使用T属性或torch.transpose方法。

### 如何使用torch.where进行条件索引？

-torch.where根据给定的条件返回满足条件的元素的索引。可以用于创建基于条件的新张量或对现有张量进行操作。

### 张量的独特值（unique）如何获取？

-可以使用torch.unique方法获取张量中所有独特的值。该方法返回一个排序后的独特值张量。

### Outlines

### 📚 深度学习基础：张量操作入门

介绍了深度学习中张量操作的重要性，强调了学习张量操作是深入理解深度学习的基础。视频将分为四个部分：张量的初始化、张量数学运算、张量索引和张量重塑。鼓励观众观看完整视频以掌握这些基础操作，即使不能全部记住，至少了解它们的存在，这将在未来节省大量时间。

### 🔢 张量初始化与属性

详细讲解了如何初始化张量，包括使用列表、指定数据类型、设置设备（CPU或CUDA）、设置梯度要求等。还介绍了如何查看张量的设备位置、数据类型、形状和是否需要梯度等属性。

### 📈 张量数学运算与比较

介绍了张量的基本数学运算，如加法、减法、除法、元素级指数运算等。还讲解了如何进行矩阵乘法、矩阵指数运算、元素级比较以及如何使用广播功能。

### 🔄 张量索引与操作

解释了如何通过索引访问和修改张量中的特定元素，包括基本索引、高级索引技巧、条件索引等。还介绍了如何使用`torch.where`进行条件赋值和`torch.unique`获取张量中的唯一值。

### 📊 张量重塑与变形

展示了如何使用`view`和`reshape`方法改变张量的形状，包括转置、展平、改变维度顺序等。强调了`view`要求张量在内存中连续存储，而`reshape`则没有这个要求。还介绍了如何使用`torch.cat`进行张量拼接。

### 🎉 总结与结束

视频总结，强调了学习张量操作的重要性，并鼓励观众在评论区提问。提醒观众，掌握这些基础操作将使后续的深度学习任务变得更加容易。

### Mindmap

### Keywords

### 💡张量（Tensor）

### 💡PyTorch

### 💡初始化（Initialize）

### 💡数学运算（Math Operations）

### 💡索引（Indexing）

### 💡重塑（Reshaping）

### 💡广播（Broadcasting）

### 💡设备（Device）

### 💡梯度（Gradient）

### 💡矩阵乘法（Matrix Multiplication）

### 💡批量矩阵乘法（Batch Matrix Multiplication）

### Highlights

介绍了张量的基本操作，强调了在深度学习中学习这些操作的重要性。

展示了如何初始化张量，包括使用列表和指定数据类型。

解释了如何设置张量在CUDA或CPU上运行。

介绍了张量的属性，如设备位置、数据类型和是否需要梯度。

提供了多种创建张量的方法，如使用torch.empty、torch.zeros、torch.rand等。

讨论了如何在不同设备之间移动张量，以及如何设置张量的设备。

展示了如何进行张量的基本数学运算，包括加法、减法和除法。

介绍了张量的广播（broadcasting）概念，以及如何在不同维度上进行操作。

解释了如何进行张量的索引，包括基本索引和高级索引技巧。

讨论了如何重塑张量，包括使用view和reshape方法。

展示了如何进行张量的矩阵乘法和矩阵指数运算。

介绍了如何进行张量的元素级操作，如元素级乘法和点积。

解释了如何使用torch.where进行条件索引和赋值。

展示了如何使用torch.unique获取张量中的唯一值。

讨论了如何使用torch.cat进行张量的拼接。

介绍了如何使用torch.permute和torch.squeeze进行张量的维度操作。

强调了学习张量操作对于未来节省时间的重要性。

提供了一个视频，旨在帮助观众掌握张量操作的基础知识。

### Transcripts

learning about the basic tensor

operation is an essential part of pi

torch and it's worth to spend some time

to learn it and it's probably the first

thing you should do before you do

anything related to deep learning what

is going on guys hope you're doing

awesome and in this video we're gonna go

through four parts so we're gonna start

with how to initialize a tensor and

there are many ways of doing that we're

gonna go through a lot of them and then

we're gonna do you know some tensor math

math and comparison operations we're

gonna go through tensor indexing and

lastly we're gonna go through tensor

reshaping and I just want to say that I

really encourage you to watch this video

to the end so you get a grasp on these

tensor operations even if you don't

memorize them after this video and

there's probably no way you can memorize

all of them you would at least know that

they exist and that will save you a lot

of time in the future so in this video I

will cover the basics but I will also go

a bit beyond that and show you a lot of

useful things or operations for

different scenarios so there's no way

I'm able to cover all of the tensor

operations there are a bunch of more

advanced ones that I don't even know

about yet but these are definitely

enough to give you a solid foundation so

I guess we'll just get started and the

first thing I'm going to show you is how

to create a tensor so what we're gonna

do is we're gonna do my tensor and we're

gonna do torch tensor and we're gonna do

the first thing we're gonna do is a list

and we're gonna do a list inside that

list we're gonna do one two three and

then let's do another list and we're

gonna do four five six so what this

means right here is that we're gonna do

two rows so this is the first row and

then this is the second row all right so

we're gonna have two rows in this case

and we're gonna have three columns

so we could do then is we can do print

my tensor and we can run that and we're

gonna just get in an in a nice format so

we're gonna get one for for the first

column two five and three six now what

we can do as well is we can set the type

of this tensor so we can do D type

equals torch dot float and we can do

let's say float 32 so then if we print

it again we're gonna get that those are

float values now another thing we can do

is we can set the device that this

tensor should be on either CUDA or the

CPU now if you have a CUDA enabled GPU

you should almost always have a sensor

on the on CUDA otherwise you're gonna

have to use the CPU but we can specify

this using the device so we can do

device equals then we can set this to

CUDA if you have that available and then

if we print my tensor again we can say

see that the device says CUDA right here

if you do not have a it could enable GPU

then you're gonna have to write CPU

right here I think that CPU is also the

default she don't have to write it but

it can help to just be specific now if

we run this the it doesn't the device

doesn't show which means that it's on

the CPU can also set other arguments

like requires gradient which is

important for auto grab which I'm not

going to cover in this video but

essentially for computing the gradients

which are used when we do the backward

propagation through the computational

graph to update our parameters through

gradient descent but anyways I'm not

gonna go in-depth on that one thing we

can do as well is that you're gonna see

a lot when in my videos and just if you

read PI torch code is that people often

write device equals CUDA if torch that

CUDA dot is available like that

else CPU all right and in this case what

happens is that if you have CUDA enabled

the device is going to be set to CUDA

and otherwise it's going to be set to

the CPU kind of that priority that if

you have enabled and you should use it

otherwise you're gonna be stuck with the

CPU but that's all you have so what we

can do instead of writing the string

here is that we can just write device

like this

and now the great thing about this is

that two people can run it and if one

has kuda it's going to run on the GPU if

they don't have it's gonna run in the

cpu but the code works no matter if you

have it or not

now let's look at some attributes of

tensors so what we can do is we can as I

said I print my tensor and we just I get

so we can do that again and we just get

some information about the tensor like

what device it's on and if it requires

gradient what we can also do is we can

do my tensor and we can do dot D type so

that would we'll just in this case print

towards up float32 and we can also do

print my tensor that device what that's

gonna do is gonna show us what there is

detention or zone so CUDA and then

you're gonna get this right here which

essentially if you have multiple GPUs

it's gonna say on which GPU it's on in

this case I only have one GPU so it's

gonna say 0 and 0 I think is the default

one if you don't specify and then what

we can do as well we can do print my

tensor dot shape so yeah this is pretty

straightforward it's just gonna print

the shape which is a two by three what

we can do as well is we can do print

might answer that requires grad which is

gonna tell us if that answer requires

gradient or not which in this case we've

set it to true alright so let's move on

to some other common initialization

methods

what we can do is if we don't have the

exact values that we want to write like

in this case we had one two three and

four five six we can do x equals torch

that empty and we can do size equals

let's say three by three and what this

is gonna do is it's gonna create a three

by three

tensor and our matrix I guess and it's

gonna be empty in that it's going to be

uninitialized data but these days this

data values that isn't gonna be in this

industry are gonna be just whatever is

in the memory at that moment so the

values can really be random so don't

think that this should be zeros or

anything like that

it's just gonna be unleash uninitialized

theta now if you would want to have

zeros you can do torched add zeros

and you don't have to specify the the

size equals since it's the first

argument we can just write three three

like that and that's gonna be so what we

can do actually we can print let's see

we can print X right here and we can see

what value is it gets and in this case

it actually got zero zeros but that's

that's not what it's gonna happen in

general and yeah if you print X after

this it's also going to be zeros now

what we can also do is we can do x

equals torch dot R and and we can do

three three again and and what this is

gonna do it it's gonna initialize a

three by three matrix with values from a

uniform distribution in the interval 0 &

1 another thing we could do is we could

do X equal torched at once I'm looking

again to I don't a by 3 and this is just

gonna be a three by three matrix with

all values of 1 another thing we can do

is torch that I and we can and this is

we're gonna send in five by five or

something like that and this is gonna

create an identity matrix so we're gonna

have ones on the diagonal and the rest

will be will be zeros so if you're

curious why it's called I is because I

like that is the identity matrix how you

write in mathematics and if you say I it

kind of sounds like I yeah that makes

sense

so anyways one more thing we can do is

we give you something like torch that a

range and we can do start we can do end

and we can do step so you know basically

the arrange function is exactly like the

range function in Python so this should

be nothing nothing weird one thing I

forgot to do is just print them so we

can see what they look like so if we

print that one as I said we're gonna

have once on the diagonal and the rest

will be 0 if we print X after the

arranged

it's going to start at zero it's going

to have a step of one and the end is a

non-inclusive v del v value or five so

we're going to have 0 1 2 3 4 so if we

print X we're gonna see exactly that 0

and then up to inclusive for another

thing we can do is we can do x equals

torsion linspace and we can specify

where it should start so we can do start

equals 0.1 and we could do something

like end equals 1 and we could also do

step equals 10 so what this is gonna do

is it's gonna write this should be steps

so what this is gonna do is it's gonna

start at 0.1 and then it's gonna end at

1 and it's gonna have 10 values in

between those so what's going to happen

in this case it's gonna take the first

value 0.1 then the next one it's going

to be a point to a point 3 0.4 etc up to

1 so just to make sure we can print X

and we see that that's exactly what

happens and if we calculate the number

of points so we have 1 2 3 4 5 6 7 8 9

10 right so we're gonna have it equal 2

steps amount of point so then we can do

also x equals torch that empty as we did

for the first one and we can set the

size and I don't know 1 and 5 or

something like that and what we can do

then is we can do dot normal and we can

do mean equals 0 and the standard

deviation equals 1 and essentially so

what this is gonna do it's gonna create

the initialized data of size 1 1 by 5

and then it's just gonna make those

values normally distribute normally

distributed with a mean of 0 and

standard deviation of 1 so we could also

do this with something like we can also

do something like with the uniform

distribution so we can do dot uniform

and then we can do 0 and 1 which would

also be similar to what we did up here

for the torch ran

but of course here you can specify

exactly what you want for the lower and

the upper of the uniform distribution

another thing we can do is to torch dot

d AG and then we can do torch dot ones

of some size let's say three it's going

to create a diagonal matrix of ones on

the diagonal it's going to be shape

three so essentially this is gonna

create a 3x3 diagonal matrix essentially

this is gonna create a an identity

matrix which is three by three so we

could adjust as well used I

but this diagonal function can be used

on on it on any matrix so that we

preserve those values across a diagonal

and in this case it's just simple to use

torched at once now one thing I want to

show as well is how to initialize tensor

to different types and how to convert

them to different types

so let's say we have some tensor and

we're just going to do a torch dot a

range of 4 so we have 0 1 2 3 and yea so

here I set to start the end and this

step similarly to Python the step will

be 1 by default and the start will be 0

by default so if you do a range for

you're just gonna do that's the end

value now I think this is this is

initialized as a 64 by default but let's

say we want to convert this into a

boolean operator so true or false what

we can do is we can do tensor dot bool

and that will just create false true

true true so the first one is 0 that's

gonna be false and the rest will be

through true now what's great about

these as I'm showing you now when you

dot boo and I'm also going to show you a

couple more is that they work no matter

if you're on CUDA or the CPU so no

matter which one you are these are great

to remember because they will always

work now the next thing is we can do

print tensor that's short and what this

is gonna do is it's gonna create two

int16

and I think both of these two are not

that often used but they're good to know

about and then we can also do tensor dot

loan and what this is gonna do is it's

gonna do it to in 64 and this one is

very important because this one is

almost always used we're going to print

tensor 1/2 and this is gonna make it to

float 16 this one is not used that often

either but if you have well if you have

newer GPUs in in the 2000 series you can

actually train your your networks on

float 16 and that's that that's when

this is used quite often but if you

don't have such a GPU I don't have that

that new of a GPU then it's not possible

to to Train networks using float 16 so

what's more common is to use tenser dot

float

so this will just be a float 32-bit and

this one is also super important this

one is used super often so it's good to

remember this one and then we also have

tensor dot double and this is gonna be

closed 64 now the next thing I'm gonna

show you is how to convert between

tensor and let's say you have a numpy

array so we'll say that we import numpy

as MP now let's say that we have some

numpy array we have numpy zeros and I'd

say we have a 5 by 5 matrix and then

let's say we want to convert this to a

tensor well this is quite easy we can do

tensor equals Torche dot from numpy and

we can just sending that numpy array

that's how we get it to a tensor now if

you want to convert it back so you have

a a back the number array we can just do

let's say numpy array back we can do

tensor dot numpy and this is gonna bring

back the number array perhaps there

might be some numerical roundoff errors

but they will be exactly identical

otherwise so that was some how to

initialize any tensor and some other

useful things like converting between

other types in float and double and also

how to convert between numpy arrays and

tensors now we're going to jump to

tensor math and comparison operations so

we're gonna first initialize two tensors

which we know exactly how to do at this

point we're going to torch that tensor

one two three and we're gonna do some

y2b torsa tensor and then I don't know

nine eight seven and

we're going to start real easy so we're

just going to start with addition now

there are multiple ways of doing

addition I'm going to show you a couple

of different points so we can do

something like is that one to be torched

empty of three values then we can do

torch that add and we can do X Y and

then we can do out equals Z one now if a

print said one we're going to get 10 10

and 10 because we've added these these

together and as we can see 1 plus 9 is

10 2 plus 8 and 3 plus 7 so this is one

way another way is to just do Z equals

torch dot add of x and y and we're gonna

get exactly the same result now another

way and this is my preferred way it's

just to do X Z equals X plus y so real

simple and real clean and you know these

are all identical so they will do

exactly the same operations and so in my

opinion there's really no way no reason

not to use just the normal addition for

subtraction there are again other ways

to do it as well but I recommend doing

it like this so we do Z equals X minus y

now for division this is a little bit

more clunky in my opinion but I think

they are doing some changes for future

versions of Pi torch but we can do Z

equals torch dot true divide and then we

can do x and y what's what's going to

happen here is that it's going to do

element wise division if they are of

equal shape so in this case it's going

to do 1/9 as its first element 2 divided

by 8 3 divided by 7 let's say that Y is

just an integer so Y is I don't know -

then what's gonna happen it's gonna

divide every element in X by 2 so it

would be 1/2 3/2 and 3/2 if Y would be

in

now another thing I'm gonna cover is in

place operations so let's say that we

have T equals towards that zeros of

three elements and let's say we want to

add X but we want to do it in place and

what that means it will mutate the

tensor in place so it doesn't create a

copy so we can do that by T dot ad

underscore X and whenever you see any

operation followed by an underscore

that's when you know that the operation

is done in place so doing these