Skip to content

model_split seems a little problem. Adam cannot get head_params, and cannot update head params!#598

Open
HaronCHou wants to merge 21 commits into
fastai:masterfrom
jav0927:master
Open

model_split seems a little problem. Adam cannot get head_params, and cannot update head params!#598
HaronCHou wants to merge 21 commits into
fastai:masterfrom
jav0927:master

Conversation

@HaronCHou

Copy link
Copy Markdown

https://github.com/jav0927/course-v3/blob/master/Fastai2_SSD_Model.ipynb
def model_split(m): return L(m.body[0], m.body[1], m.body[2], m.body[3], m.body[4], m.body[5], m.body[6], m.body[7], m.drop).map(params)

epoch train_loss valid_loss time
0 48.916576 47.567001 00:55
1 48.897285 47.704815 00:54
2 48.736855 47.622379 00:54

if I change model_split as follows:

def model_split(m): body_params, head_params = L(),L() # body_params = L(m.body[0], m.body[1], m.body[2], m.body[3], m.body[4], m.body[5], m.body[6], m.body[7], m.drop).map(params) # head_params = L(m.sconv0, m.sconv1, m.sconv2, m.sconv3, m.out0, m.out1, m.out2, m.out3).map(params) for p in m.body: body_params += params(p) head_params += params(m.sconv0) head_params += params(m.sconv1) head_params += params(m.sconv2) head_params += params(m.sconv3) head_params += params(m.out0) head_params += params(m.out1) head_params += params(m.out2) head_params += params(m.out3) return L(body_params, head_params)

Then, loss like:
​​​​
image

I think probably this is a problem

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants