Restoring data with restore-workspace.sh not working

[AungDev]

Get stuck in restoring data to self-hosted Huly using ./restore-workspace.sh. Any suggestion/guide is appreciated.

Huly is setup on Ubuntu by

git clone https://github.com/hcengineering/huly-selfhost.git
cd huly-selfhost
./setup.sh --quick

• Huly is working. I can create and login with new user account.

Followed the steps in https://github.com/hcengineering/platform/blob/develop/docs/guides/backup-restore.en.md

• Downloaded backup using on click download backup button.

• Extracted backup.zip file.

• Running "./restore-workspace.sh ./path_to/backups/myws myws" show no message. I tried to find if restoring backup successful or not but no clue yet.

[huly-github-staging]

Connected to Huly®: SELFH-309

[AungDev]

Workaround is to use "backup-restore.sh" instead of "restore-workspace.sh".

[mc-putchar]

Script fails at this instruction:

huly-selfhost/restore-workspace.sh

Line 89 in e8c0350

PASSWORD="$(LC_ALL=C tr -dc 'A-Za-z0-9' </dev/urandom | head -c 16)"

Because head drops out as soon as it's done, and tr throws SIGPIPE activating the pipefail exit of the script.
Quickfix would be to wrap the call with pipefail deactivated:

set +o pipefail
PASSWORD="$(LC_ALL=C tr -dc 'A-Za-z0-9' </dev/urandom | head -c 16)"
set -o pipefail

[ArtyomSavchenko]

Hi
I applied a fix for this script.

[mc-putchar]

Thank you for fixing that.

I still ran into some issues in the script to execute proper workspace migration:

• $WORKSPACE (the second arg) is used both as workspace name to create a workspace (which creates it with a new uuid), and later where id is expected (where the newly created workspace-id is not used).
• create-workspace drops the id instead of using it for subsequent actions
• assign-workspace takes email then workspace-id, script has those args inverted
  

  huly-selfhost/restore-workspace.sh

  Line 124 in 9565090

  if ! ./run-tool.sh assign-workspace "$WORKSPACE" "$EMAIL"; then

I proposed a fix with PR #313

---

However, I ran into a bigger issue while running the backup-restore.sh part of it, where the user accounts would not get migrated. This error appeared in the logs:

collected account objects domain="account.person" count=4 workspace="REDACTED"
failed to process domain err={"message":"null value in column \"last_name\" violates not-null constraint","stack":"PostgresError: null value in column \"last_name\" violates not-null constraint
    at ErrorResponse (/usr/src/app/bundle.js:241468:31)
    at handle (/usr/src/app/bundle.js:241225:11)
    at Socket.data2 (/usr/src/app/bundle.js:241034:13)
    at Socket.emit (node:events:519:28)
    at addChunk (node:internal/streams/readable:561:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
    at Readable.push (node:internal/streams/readable:392:5)
    at TCP.onStreamRead (node:internal/stream_base_commons:189:23)","name":"PostgresError","severity_local":"ERROR","severity":"ERROR","code":"23502","file":"errors.go","line":"68","routine":"NewNonNullViolationError"} domain="account.person" workspaceId="REDACTED"

Inspecting the JSONs packed in account.person-data of backup, most accounts have lastName:"", but some (that used GH OAuth, I suspect) had it set to null.

In case someone else runs into the same issue:

# Pass the archive name as argument
# e.g. account.person-data-<uuid>.tar.gz
ARCHIVE=$(realpath "$1")

TEMP_DIR=$(mktemp -d)
trap 'rm -fr $TEMP_DIR' EXIT

tar xzf "$ARCHIVE" -C "$TEMP_DIR"

find "$TEMP_DIR" -type f -name "*.json" -exec sed -i 's/"lastName":null/"lastName":""/g' {} +

pushd "$TEMP_DIR" || exit
tar czf "$ARCHIVE" ./*
popd