TECHNICAL FORUM
Smart Sharing | [Lecture Series on Artificial Intelligence Methods and Applications] Lecture 3- Deep Learning
September 10, 2021
Author: Deer Smart
View: 316
Deep learning
Artificial intelligence is a strategic technology that leads the future. Understand the development process and current situation of the field of artificial intelligence, as well as the basic principles of machine learning, neural networks, and deep learning
Applicable scenarios, algorithmic methods, and through course learning, one can use artificial intelligence thinking to think and solve practical problems in combination with their own work.
The series of lectures on artificial intelligence methods and applications (which will be divided into four chapters: the first, second, third, and fourth) will be gradually shared with everyone. The speaker is a PhD from Hefei University of Technology
Postdoctoral researcher Lv Zengwei's comprehensive understanding of artificial intelligence is a pure sharing that is worth learning and collecting. (Thanks: Due to the reprinting or adaptation of reference sources, the copyright belongs to the original author. We would like to express our gratitude to the original author!).
This is the third lecture - Deep Learning.
1、 Principles of Deep Learning
What is depth? Why depth?
Deep=Many hidden layers
Advantages: 1. High efficiency. 2. Plasticity. If traditional algorithms are used to solve a problem, the cost of adjusting the model may be rewriting the code, which makes the cost of improvement enormous.
Deep learning only requires adjusting parameters to change the model. 3. Universality. Neural networks solve problems through learning and can automatically build models based on the problem,
So it can be applied to various problems, rather than limited to a fixed problem.
Disadvantages: 1. High training costs. Deep learning requires the use of high-end GPUs. These GPUs are very expensive, but without them, it would be difficult to achieve high-performance deep networks.
To effectively utilize such high-end GPUs, a fast CPU, SSD storage, and fast and large capacity RAM are also required. The classic ML algorithm can be trained well with just a decent CPU and does not require the best hardware.
2. Will not directly learn knowledge. Knowledge is extracted from data. If you want to teach human knowledge directly to it, such as Einstein's theory of relativity, then there is currently no corresponding solution.
3. Not good at solving certain specific problems. The use of neural networks for complex mathematical operations may not be the optimal solution, and other algorithms need to be combined. 4. The essence is a black box, and its theory is incomplete.
Since the depth is good, why not the depth?
Gradient vanishing and gradient explosion problems
Essentially, it is the same, all due to the multiplication effect in gradient backpropagation caused by too many layers in the network.
In scalar cases, either when the absolute value of this value is greater than 1, a gradient explosion occurs; Either when the absolute value of this value is less than 1, gradient disappearance occurs until it reaches 0.
The progress in the field of deep learning mainly relies on the increasing amount of training data and computing power (the development of CPU and GPU); A huge breakthrough was the use of the relu function instead of the sigmoid function.
2、 Convolutional neural network
Artificial intelligence is a strategic technology that leads the future. Understand the development process and current situation of the field of artificial intelligence, as well as the basic principles of machine learning, neural networks, and deep learning
Applicable scenarios, algorithmic methods, and through course learning, one can use artificial intelligence thinking to think and solve practical problems in combination with their own work.
The series of lectures on artificial intelligence methods and applications (which will be divided into four chapters: the first, second, third, and fourth) will be gradually shared with everyone. The speaker is a PhD from Hefei University of Technology
Postdoctoral researcher Lv Zengwei's comprehensive understanding of artificial intelligence is a pure sharing that is worth learning and collecting. (Thanks: Due to the reprinting or adaptation of reference sources, the copyright belongs to the original author. We would like to express our gratitude to the original author!).
This is the third lecture - Deep Learning.
1、 Principles of Deep Learning
What is depth? Why depth?
Deep=Many hidden layers
Advantages: 1. High efficiency. 2. Plasticity. If traditional algorithms are used to solve a problem, the cost of adjusting the model may be rewriting the code, which makes the cost of improvement enormous.
Deep learning only requires adjusting parameters to change the model. 3. Universality. Neural networks solve problems through learning and can automatically build models based on the problem,
So it can be applied to various problems, rather than limited to a fixed problem.
Disadvantages: 1. High training costs. Deep learning requires the use of high-end GPUs. These GPUs are very expensive, but without them, it would be difficult to achieve high-performance deep networks.
To effectively utilize such high-end GPUs, a fast CPU, SSD storage, and fast and large capacity RAM are also required. The classic ML algorithm can be trained well with just a decent CPU and does not require the best hardware.
2. Will not directly learn knowledge. Knowledge is extracted from data. If you want to teach human knowledge directly to it, such as Einstein's theory of relativity, then there is currently no corresponding solution.
3. Not good at solving certain specific problems. The use of neural networks for complex mathematical operations may not be the optimal solution, and other algorithms need to be combined. 4. The essence is a black box, and its theory is incomplete.
Since the depth is good, why not the depth?
Gradient vanishing and gradient explosion problems
Essentially, it is the same, all due to the multiplication effect in gradient backpropagation caused by too many layers in the network.
In scalar cases, either when the absolute value of this value is greater than 1, a gradient explosion occurs; Either when the absolute value of this value is less than 1, gradient disappearance occurs until it reaches 0.
The progress in the field of deep learning mainly relies on the increasing amount of training data and computing power (the development of CPU and GPU); A huge breakthrough was the use of the relu function instead of the sigmoid function.
2、 Convolutional neural network
