Multiple parse errors caused in import and export syntax

[tekacs]

When trying to convert typings for libraries like Vue or Vuetify, issues like the following seems to arise:

Vue, converting this file

amar@tekpro ~/repos/ext/scala-js-ts-importer master $ sbt 'run node_modules/vue/types/index.d.ts Vue.scala vue'
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/amar/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/amar/repos/ext/scala-js-ts-importer/project
[info] Loading settings from build.sbt ...
[info] Set current project to scala-js-ts-importer (in build file:/Users/amar/repos/ext/scala-js-ts-importer/)
[info] Running org.scalajs.tools.tsimporter.Main node_modules/vue/types/index.d.ts Vue.scala vue
Parse error at 3.8
'identifier as' expected but `default' found
export default Vue;
       ^

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "run-main-0"
[error] java.lang.RuntimeException: Nonzero exit code: 2
[error] 	at sbt.Run$.executeTrapExit(Run.scala:124)
[error] 	at sbt.Run.run(Run.scala:77)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1168)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1163)
[error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:366)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at scala.util.Try$.apply(Try.scala:209)
[error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:289)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] 	at java.base/java.lang.Thread.run(Thread.java:844)
[error] (Compile / run) Nonzero exit code: 2
[error] Total time: 2 s, completed May 4, 2018, 9:25:52 AM

It looks like export default syntax isn't supported in the parser.

Also Vue, trying this file instead

amar@tekpro ~/repos/ext/scala-js-ts-importer master $ sbt 'run node_modules/vue/types/vue.d.ts Vue.scala vue'
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/amar/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/amar/repos/ext/scala-js-ts-importer/project
[info] Loading settings from build.sbt ...
[info] Set current project to scala-js-ts-importer (in build file:/Users/amar/repos/ext/scala-js-ts-importer/)
[info] Running org.scalajs.tools.tsimporter.Main node_modules/vue/types/vue.d.ts Vue.scala vue
Parse error at 14.1
IdentifierName expected
} from "./options";
^

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "run-main-0"
[error] java.lang.RuntimeException: Nonzero exit code: 2
[error] 	at sbt.Run$.executeTrapExit(Run.scala:124)
[error] 	at sbt.Run.run(Run.scala:77)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1168)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1163)
[error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:366)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at scala.util.Try$.apply(Try.scala:209)
[error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:289)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] 	at java.base/java.lang.Thread.run(Thread.java:844)
[error] (Compile / run) Nonzero exit code: 2
[error] Total time: 2 s, completed May 4, 2018, 9:18:05 AM

In this case it looks like the use of trailing commas in the import list isn't being handled.

Vuetify:

amar@tekpro ~/repos/ext/scala-js-ts-importer master $ sbt 'run node_modules/vuetify/index.d.ts Vuetify.scala vuetify'
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/amar/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/amar/repos/ext/scala-js-ts-importer/project
[info] Loading settings from build.sbt ...
[info] Set current project to scala-js-ts-importer (in build file:/Users/amar/repos/ext/scala-js-ts-importer/)
[info] Running org.scalajs.tools.tsimporter.Main node_modules/vuetify/index.d.ts Vuetify.scala vuetify
Parse error at 2.1
'`;'' expected but `import' found
import 'vuetify/src/util/colors'
^

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "run-main-0"
[error] java.lang.RuntimeException: Nonzero exit code: 2
[error] 	at sbt.Run$.executeTrapExit(Run.scala:124)
[error] 	at sbt.Run.run(Run.scala:77)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1168)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1163)
[error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:366)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at scala.util.Try$.apply(Try.scala:209)
[error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:289)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] 	at java.base/java.lang.Thread.run(Thread.java:844)
[error] (Compile / run) Nonzero exit code: 2
[error] Total time: 2 s, completed May 4, 2018, 9:17:43 AM

This case is complaining about the lack of semicolons after imports - inserting those redundant semicolons however, results in another form of import syntax failing:

amar@tekpro ~/repos/ext/scala-js-ts-importer master $ sbt 'run node_modules/vuetify/index.d.ts Vuetify.scala vuetify'
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/amar/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/amar/repos/ext/scala-js-ts-importer/project
[info] Loading settings from build.sbt ...
[info] Set current project to scala-js-ts-importer (in build file:/Users/amar/repos/ext/scala-js-ts-importer/)
[info] Running org.scalajs.tools.tsimporter.Main node_modules/vuetify/index.d.ts Vuetify.scala vuetify
Parse error at 3.11
'identifier from' expected but `,' found
import Vue, { PluginFunction } from 'vue';
          ^

Exception: sbt.TrapExitSecurityException thrown from the UncaughtExceptionHandler in thread "run-main-0"
[error] java.lang.RuntimeException: Nonzero exit code: 2
[error] 	at sbt.Run$.executeTrapExit(Run.scala:124)
[error] 	at sbt.Run.run(Run.scala:77)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5(Defaults.scala:1168)
[error] 	at sbt.Defaults$.$anonfun$bgRunTask$5$adapted(Defaults.scala:1163)
[error] 	at sbt.internal.BackgroundThreadPool.$anonfun$run$1(DefaultBackgroundJobService.scala:366)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at scala.util.Try$.apply(Try.scala:209)
[error] 	at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:289)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] 	at java.base/java.lang.Thread.run(Thread.java:844)
[error] (Compile / run) Nonzero exit code: 2
[error] Total time: 1 s, completed May 4, 2018, 9:22:04 AM

The implementation of import syntax seems to be fairly distant from the possibilities in current versions of TypeScript.

I'm happy to update the parser to support more of these cases - that said however, I'm hesitant to make updates if it might prove to be a recurring theme elsewhere that TypeScript syntax isn't fully captured.

Any thoughts on how this could proceed?

---

Aside:

Given that dependencies only point to scala-parser-combinators and scalatest, might a more maintainable option be to look into targeting Scala.js and using TypeScript's own compiler interface for parsing instead (similarly to e.g. https://github.com/rrdelaney/ReasonablyTyped)? A stretch would be scalameta output, too. This might be more in the scope of a wholly new project, however.

[sjrd]

I'm happy to update the parser to support more of these cases - that said however, I'm hesitant to make updates if it might prove to be a recurring theme elsewhere that TypeScript syntax isn't fully captured.

Any thoughts on how this could proceed?

Fixing the parser is the way forward in the context of the approach taken by this repository, yes. There might be follow-up issues, of course. But PRs that improve the support of input TypeScript files are always welcome.

If you feel like changing completely the architecture and use TypeScript's own compiler, that's fine, but I cannot guarantee that I will upstream such dramatic changes. Forks are not a bad thing anyway.

[tekacs]

Fixing the parser is the way forward in the context of the approach taken by this repository, yes. There might be follow-up issues, of course. But PRs that improve the support of input TypeScript files are always welcome.

Understood - I might take a look at trying to fix all of the issues that I can find in Vue and Vuetify above in the parser.

Depending on the result of that (and trying with a few other libraries that serve as realistic 'sample' inputs), it might be worth exploring the option of changing things up somewhat (if time allows).

Thanks for getting back so quickly!