Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 40 additions & 6 deletions src/Feldspar/Onnx/onnxToFeld.hs
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,49 @@ import qualified Data.ByteString.Builder as B

import qualified Data.Foldable as D (toList, foldMap)
import qualified Data.Set as S
import Control.Monad (when)
import Data.List (intercalate)
import Data.Maybe (fromJust, fromMaybe)
import System.Console.GetOpt
import System.Environment (getArgs)
import System.Exit (die, exitSuccess)
import System.FilePath (takeBaseName, (<.>))
import System.IO (IOMode(WriteMode), BufferMode(BlockBuffering), openFile, hClose, hPutStr
, hSetBuffering, hSetBinaryMode)

-- | Type of options record
data Options = Options { writeInit :: Bool
, showHelp :: Bool
}

-- | Default options
defaultOptions :: Options
defaultOptions = Options
{ writeInit = False
, showHelp = False
}

-- | Option descriptons
optDescrs :: [OptDescr (Options -> Options)]
optDescrs
= [ Option "w" ["writeInits"] (NoArg $ \ o -> o{writeInit = True}) "Write model initializers to file"
, Option "h" ["help"] (NoArg $ \ o -> o{showHelp = True}) "Show usage info"
]

-- | Usage info
usageStr :: String
usageStr = usageInfo "Usage: onnxToFeld <options> <model>" optDescrs

-- | Main entry point
main :: IO ()
main = do args <- getArgs
let [modelFileName] = take 1 args -- First argument is file name
let (fs, nonOpts, errs) = getOpt RequireOrder optDescrs args
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think onnxToFeld should be viewed as a part of "Feldspar". With that perspective, it would make sense to have the pass options and everything else that the option handling in Feldspar implements. Can't we just add writeInits to the Options in Feldspar.Compiler.Options, and import most of the option handling logic from Feldspar.Compiler here?

(I realize this makes writeInits visible when just using the Feldspar library and the option wouldn't do anything useful there. I care less about that interface since it's not a user-facing interface.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I want the plumbing in place from the beginning is that I'm scared, the plumbing has taken a lot of effort to bolt on afterwards in the library (and even with that effort, the result is still not great).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added error checking to compileC in Plugin and started using that code in the RegressionTests instead of having duplicate code in the test suite for that. Can we get onnxToFeld to compile everything all the way to an executable by just ending main with:

compile opts ...
compileC opts cname oname []

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more complicated than that, since onnxToFeld produces Faldpar source code, so you also need to run ghc on that and then run the resulting program with e.g. readProcessWithExitCode and then you could do compileC.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, one more level in the tower, and I think when running the resulting program is where compileC is called (perhaps by a regular pass using the pass manager?).

I assume I wasn't very clear, my suggestion for this patch was just to consolidate the option handling so we have one option handling to put us in a good position for taking the steps you outlined at a later time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think onnxToFeld will ever need multiple passes, so the PassMgr is beside the point. Apart from that onnxToFeld currently has a '-h' option and might perhaps get a '-o' option or so in the future, but the bulk of its options will probably be specific to its function. Possibly we could have a layer on top of GetOpt dealing with common options; I think that would be better than using the same option type for onnxToFeld and Feldspar compiler.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we pass --writeAfter FPUnASTF along to the compiled Feldspar program from inside onnxToFeld in the future?

opts = foldl (flip ($)) defaultOptions fs
when (not (null errs) || null nonOpts) $ die usageStr
when (showHelp opts) $ do
putStr usageStr
exitSuccess
let [modelFileName] = take 1 nonOpts -- First argument is file name
modelBaseName = takeBaseName modelFileName
dataFileName = modelBaseName <.> "data"
progFileName = modelBaseName <.> "hs"
Expand All @@ -70,11 +103,12 @@ main = do args <- getArgs
gr = fromMaybe (error "No graph in model") $ O.graph model

-- Write the weights
dfile <- openFile dataFileName WriteMode
hSetBinaryMode dfile True
hSetBuffering dfile $ BlockBuffering Nothing
B.hPutBuilder dfile $ D.foldMap buildInitTensor $ G.initializer gr
hClose dfile
when (writeInit opts) $ do
dfile <- openFile dataFileName WriteMode
hSetBinaryMode dfile True
hSetBuffering dfile $ BlockBuffering Nothing
B.hPutBuilder dfile $ D.foldMap buildInitTensor $ G.initializer gr
hClose dfile

-- Write the program
pfile <- openFile progFileName WriteMode
Expand Down