Sets for hyperparameter search

[agadetsky]

Could you please share hyperparameter sets that were considered and tried during Graph Layout experiment for each of the SST?

[agadetsky]

@choidami @mbp28 I can't reproduce your results for undirected spanning trees at the moment. Can you help with hyperparameters search space for T=10 or T=20? I tried lrs from 1e-3 to 1e-5 and temps from 0.1 to 10 in logscale. Any of that gives very bad results. Seems like encoder stucks in bad local optima and can't produce sought latent spanning trees. Maybe you have left some checkpoints of trained models with good hyperparameters? Thank you in advance!

[choidami]

Hi,
I'm sorry for the late response. When we tuned for spanning tree, our learning rate range was [1e-4, 1e-2] (and temperature [1e-1, 1e1] which is what you already have). I'm pretty confident you can get good results with [1e-5, 1e-3] though. We did random search with 20 trials, maybe you need to run more trials?
I was able to get a successful run with these hyperparameters:
lr = 0.0012429952432139483
temp = 3.863964701119693
seed = 42

[agadetsky]

@choidami thank you for your response. This hyperparamer set worked. Can you please share successful hyperparameters for undirected spanning tree for T=20?

[choidami]

I'm glad it worked!
For T=20 it should be easier to find a working set of hyperparameters.
One of the trials that worked for us was:
lr = 0.00019936764261965588
temp = 0.34170753080211275