Update the endpoint for custom fields addition and add option to allow custom fields with non-ascii characters

[sergioamorim]

The endpoint /api/customField is used on newer versions of TheHive to add custom fields. Also, to add custom fields that have non-ascii characters, the option ensure_ascii on json.dumps needs to be set to False (it is True by default).

[fjavierc]

Hi Sergioamorim,

Thank you very much for your PR, it took days trying to import "custom fields" and although I had seen that it is now "customField" I had not been able to import anything.

In my case, when using the submit_config.py script, I get the following

thehive@ubuntu:~/ManageConfig.fix$ ./submit_config.py 
Traceback (most recent call last):
  File "./submit_config.py", line 102, in <module>
    main()
  File "./submit_config.py", line 96, in main
    args = parse_args()
  File "./submit_config.py", line 90, in parse_args
    action=argparse.BooleanOptionalAction

With a small change in the script, I have been able to import correctly although I have yet to see the real scope of that change in my lab.

thehive@ubuntu:~/ManageConfig.fix$ ./submit_config.py 
usage: submit_config.py [-h] -k KEY -u URL -c CONFIG_PATH [--utf-json UTF_JSON]
submit_config.py: error: the following arguments are required: -k/--key, -u/--url, -c/--config_path

thehive@ubuntu:~/ManageConfig.fix$ ./submit_config.py -k xxxxxxxxx -u http://127.0.0.1:9000/thehive -c thehivecustomfield.conf --utf-json false
Submitting config to  http://127.0.0.1:9000/thehive
Inserting observable  datatypes...
Inserting custom fields...
Inserting metrics...
Inserting case templates...

The change I have made has been to comment a couple of lines

def parse_args():
   [...]
    parser.add_argument(
        '--utf-json',
        dest='utf_json',
        help='Do not force ASCII decoding of the configuration file (only for custom fields)'
#        help='Do not force ASCII decoding of the configuration file (only for custom fields)',
#        action=argparse.BooleanOptionalAction
    )
    return parser.parse_args()

I'll tell you about it in case you can tell me what could have failed me or if the small change is enough.

Regards,

[sergioamorim]

@fjavierc thanks for reaching out!

I am sorry, I should have mentioned this before: the BooleanOptionalAction action from argparse only became available on the version 3.9 of Python. If you are using an older version, that is probably what is causing that error on line 90. Can you try again with a newer version, please?

Also, I see you have set --utf-json false - this argument is meant to be optional, so when you don't want it to be set you simply don't write it... It will probably force you to use it since the line action=argparse.BooleanOptionalAction is commented, but that is not intended. I suggest not to use the --utf-json unless there are problems related with the encoding of the custom fields (like strange characters being shown on the UI after importing them).

[fjavierc]

Sergioamorim,

Tested with python 3.9 and everything correct !! I appreciate that you have bothered to answer me and for the clarification of my doubt.

thehive@ubuntu:~/ManageConfig.fix$ python3.9 submit_config.py 
usage: submit_config.py [-h] -k KEY -u URL -c CONFIG_PATH [--utf-json | --no-utf-json]
submit_config.py: error: the following arguments are required: -k/--key, -u/--url, -c/--config_path

Regards