Skip to content

Add sigmoid activation function to Value class #1

Description

@mihaicodes

Currently, Fermigrad supports activations like ReLU and tanh, but the original not published picograd I was working on had more activation functions, like sigmoid. It should be added back to support more neural network use cases.

Proposed Implementation:

def sigmoid(self):
    s = 1 / (1 + math.exp(-self.data))
    out = Value(s, (self,), 'sigmoid')

    def _backward():
        self.grad += s * (1 - s) * out.grad
    out._backward = _backward

    return out

Tasks:

  • Implement the sigmoid method in the Value class for the Python backend.
  • Add unit tests to verify forward pass correctness.
  • Add tests for backward pass (gradients).
  • If applicable, add C++ backend equivalent.
  • Update documentation to include sigmoid.

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions