Правильно ли у меня написанны алгоритмы прямого и обратного распостранения для нейроной сети на Python?
Я решил написать с нуля (в целях обучения и понимания темы) пару классов которые должны в совокупности представлять нейронную сеть. Вопрос заключается в том правильно ли я написал алгоритмы для прямого и обратного распространения сети (смещения я пока не добавлял намеренно)?
Библиотеки:
import typing
import numpy
import numpy as np
Класс который представляет собой нейрон:
class HiddenNeuron:
def __init__(self, vector_weight: np.ndarray, index_in_layer: int = None, index_in_site: int = None):
self.__weight_vector = vector_weight
self.__weighted_sum = None
self.__active_value = None
self.__index_in_layer = index_in_layer
self.__index_in_site = index_in_site
@property
def index_in_layer(self):
return self.__index_in_layer
@property
def index_in_site(self):
return self.__index_in_site
@property
def weight_vector(self):
return self.__weight_vector
def set_weight_value(self, index, value):
self.__weight_vector[index] = value
def weigh_sum(self, input_vector: np.ndarray) -> np.ndarray:
self.__weighted_sum = np.dot(input_vector, self.__weight_vector)
return self.weighted_sum
def activate_neuron(self, func: typing.Callable) -> float:
if self.__weighted_sum is None:
raise ErrorRoJQ("Вы пытаетесь взвесить пустую сумму!")
self.__active_value = func(self.__weighted_sum)
return self.__active_value
@property
def weighted_sum(self) -> None | np.ndarray:
return self.__weighted_sum
@property
def active_value(self) -> None | float:
return self.__active_value
Класс который представляет собой скрытый слой:
class HiddenLayer:
def __init__(self, number_of_neurons: int, weight_max: int, input_vector_len: int, index_in_site: int = None):
self.__input_vector_len = input_vector_len
self.__neuron_list: list[HiddenNeuron] = []
self.__neuron_active_values: None | list = []
self.__neuron_weight_sum: None | list = []
self.__index_in_site = index_in_site
for index_in_layer in range(number_of_neurons):
vector_weights = np.array([np.random.random() * weight_max for i in range(input_vector_len)])
new_neuron = HiddenNeuron(vector_weights, index_in_layer, index_in_site)
self.__neuron_list.append(new_neuron)
def __getitem__(self, item: int):
return self.__neuron_list[item]
def weigh_sum(self, input_vector: np.ndarray):
if len(input_vector) != self.__input_vector_len:
raise ErrorRoJQ("Длинна входного вектора, не совпадает с указаной длиной вектора весов для данного слоя!")
new_neuron_list = []
self.__neuron_weight_sum =[]
for neuron_obj in self.__neuron_list:
weight_sum = neuron_obj.weigh_sum(input_vector)
self.__neuron_weight_sum.append(weight_sum)
new_neuron_list.append(neuron_obj)
self.__neuron_list = new_neuron_list
def activate_layer(self, activate_func: typing.Callable):
new_neuron_list = []
self.__neuron_active_values = []
for neuron_obj in self.__neuron_list:
active_value = neuron_obj.activate_neuron(activate_func)
self.__neuron_active_values.append(active_value)
new_neuron_list.append(neuron_obj)
self.__neuron_list = new_neuron_list
def weigh_sum_and_activate_layer(self, input_vector, activate_func):
self.weigh_sum(input_vector)
self.activate_layer(activate_func)
@property
def neurons_active_values(self):
return self.__neuron_active_values
@property
def neurons(self):
return self.__neuron_list
@property
def neurons_weight_sum(self):
return self.__neuron_weight_sum
def __len__(self):
return len(self.__neuron_list)
Класс который представляет собой саму сеть:
class NeuronSite:
def __init__(self, hidden_layers: int,
hidden_layer_length: int,
weights_max: int,
input_vector_len: int,
output_vector_len: int):
def generate_hidden_layer(vector_len_: int, neurons_in_layer: int, index_in_site: int):
new_layer = HiddenLayer(neurons_in_layer, weights_max, vector_len_, index_in_site)
return new_layer
self.__hidden_layers: list[HiddenLayer] = []
self.__input_vector_len = input_vector_len
self.__output_vector_len = output_vector_len
vector_len = input_vector_len
# Генерация скрытых слоев
for i in range(hidden_layers):
self.__hidden_layers.append(generate_hidden_layer(vector_len, hidden_layer_length, i))
vector_len = hidden_layer_length
self.__hidden_layers.append(generate_hidden_layer(vector_len, output_vector_len, hidden_layers))
def forward_direction(self, input_vector: np.ndarray, activate_func: typing.Callable):
current_input_vector = input_vector
if len(input_vector) != self.__input_vector_len:
raise ErrorRoJQ("Входной вектор имеет не ту длину!")
for hidden_layer in self.__hidden_layers:
hidden_layer.weigh_sum_and_activate_layer(current_input_vector, activate_func)
current_input_vector = hidden_layer.neurons_active_values
output_value = self.__hidden_layers[-1].neurons_active_values
return output_value
def backpropagation(self, out_vector: list, learn_step: float, need_vector: np.ndarray):
def move_weight(neuron_: HiddenNeuron):
for weight_index, weight_value in enumerate(neuron_.weight_vector):
back_layer_index = neuron_.index_in_site - 1
back_layer = self.__hidden_layers[back_layer_index]
gradient_descent = (neuron_.weighted_sum - need_vector.sum()) * back_layer.neurons[weight_index].active_value
new_weight = weight_value - learn_step * gradient_descent
neuron_.set_weight_value(weight_index, new_weight)
cost_value = RootMeanSquareEstimation.function(need_vector.sum(), sum(out_vector))
print(f"Значение функции ошибки: {cost_value}")
for hidden_layer in self.__hidden_layers:
for neuron in hidden_layer.neurons:
move_weight(neuron)
return cost_value
Класс который используется для оценки(CostFunction):
class RootMeanSquareEstimation:
@staticmethod
def function(y: float, x: float):
"""
:param y: need_vector sum
:param x: out_vector sum
:return:
"""
return 0.5 * (y - x)**2
@staticmethod
def derivative_x(y, x):
return x - y
@staticmethod
def derivative_y(y, x):
return y - x