Add `--load` CLI flag to design for `create-request`

[joelostblom]

This was decided in #43 to be used for launching the TUI with an existing request.yaml file. Would that imply that we make SOURCE optional?

[lwjohnst86]

Ah, I thought source was optional. Yes, for sure 👍

[joelostblom]

Yes, it's currently optional in the sense that it can be left unspecified on the CLI, but then it receives the default value of datapackage.json in the current directory. Designing --load request.yaml would imply that when --load is used, either no default value is assigned to SOURCE or the default value is ignored. Do you have a preference or should we leave this decision until the implementation stage?

Writing this, I thought that another alternative could be that request.yaml could also be passed as SOURCE. So both of these would be valid:

create-request datapackage.json

create-request request.yaml

It does make SOURCE more complicated though; what do you prefer?

[lwjohnst86]

Hm, good design is about predictability, and having the first position be for two different things is unpredictable behaviour. I like the create-request --load datapackage.json idea 👍

If --load is used, SOURCE needs to point to the metadata file that the request is based on. Whether that is the default or whether that is explicitly given doesn't matter, it needs to be the same in order for the request creation to work.

[joelostblom]

If --load is used, SOURCE needs to point to the metadata file that the request is based on.

Interesting, I was imagining that the call would just be

create-request --load request.yaml

But you are saying that you prefer the following:

create-request path/to/datapackage.json --load request.yaml

What would be the benefits of requiring the datapackage.json file with --load? I imagined it would be simpler if load just took a request.yaml and did not read any datapackage file. And how would conflicting info be handled; an error to be resolved manually or an option to update the request file from the datapackage?

[lwjohnst86]

create-request --load request.yaml

This is correct, that's what I was also saying. I was adding that the default datapackage.json in that case would have to have been the same datapackage that was used for the request.yaml ☺️

[lwjohnst86]

In order for create-request to work correct if loading an existing request.yaml, it still needs a datapackage.json file to check that the request.yaml file is correct (both during loading but also during editing/saving).

[joelostblom]

it still needs a datapackage.json file to check that the request.yaml file is correct (both during loading but also during editing/saving).

Ah ok, I see, I was thinking it could use the already specified datapackage from the request.yaml, but then we would also have to store the location of that datapackage which we currently don't. Continuing that thought: Do you think it would be valuable to the owner to know the location the datapackage metadata was loaded from? I'm thinking in case it would help either statistical purposes in garden ("most people download and read data locally..") or troubleshooting (e.g. finding out where a data package which out of date information was loaded from so that it can be updated)?

What do you think the behavior should be when there is a mismatch between the info saved in request.yaml and the specified datapacakge.json? I'm thinking just outputting the values of the mismatched fields for both would be enough.

[lwjohnst86]

Related to the Owner, we'd recommend (as detailed in the package layout chapter of the data-pkg-guide) that all requests are stored in the same repo as the data package, so no, I don't think we need to include where the data package came from (plus, it might be a bit hard to do, if it was e.g. a clone of a git repo, would the location be the users path? it gets complicated fast).

If there is a mismatch, it's the request that is out of date and needs to be corrected. We should assume the metadata file is the source of truth. I'm not sure I understand that sentence though, what do you mean?

[joelostblom]

If there is a mismatch, it's the request that is out of date and needs to be corrected. We should assume the metadata file is the source of truth. I'm not sure I understand that sentence though, what do you mean?

Makes sense and it sounds like you understood what I meant. My last sentence was asking if the output of the error message would look something like this:

Error: Mismatch in the following fields of the data package:

request.yaml
version 1.2.3

datapackage.json
version 1.3.0

Do you want to automatically update request.yaml with the value from datapackage.json? [Y/n]

[lwjohnst86]

Yea, that sounds good 👍