MREF installation fails with 19.6.0 and 19.5.1 versions of mascli for 9.2-feature release

[vsolasa]

MAS CLI version

19.6.0

CLI function used

install

What happened?

MREF installation fails using mascli 19.6.0

Relevant log output

[ibmmas/cli:19.6.0]mascli$ mas install

IBM Maximo Application Suite Admin CLI v19.6.0
Powered by https://github.com/ibm-mas/ansible-devops/ and https://tekton.dev/


1) Set Target OpenShift Cluster
Already connected to OCP Cluster:
 https://console-openshift-console.apps.itz-voadf3.infra02-lb.dal12.techzone.ibm.com

Proceed with this cluster? [y/n] y

2) Choose Install Mode
There are two flavours of the interactive install to choose from: Simplified and Advanced.  The simplified option will present fewer dialogs, but you lose the ability to configure the following aspects of the installation:
 - Configure installation namespaces
 - Provide pod templates
 - Configure Single Sign-On (SSO) settings - Configure whether to trust well-known certificate authorities by default (defaults to enabled)
 - Configure whether the Guided Tour feature is enabled (defaults to enabled)
 - Configure whether special characters are allowed in usernames and userids (defaults to disabled)
 - Configure a custom domain, DNS integrations, routing mode and manual certificates
 - Customize Maximo Manage database settings (schema, tablespace, indexspace)
 - Customize Maximo Manage server bundle configuration (defaults to "all" configuration)
 - Enable optional Maximo Manage integration Cognos Analytics and Watson Studio Local
 - Enable optional Real Estate and Facilities configurations
 - Customize Db2 node affinity and tolerations, memory, cpu, and storage settings (when using the IBM Db2 Universal Operator)
 - Choose alternative Apache Kafka providers (default to Strimzi)
 - Customize Grafana storage settings
Show advanced installation options? [y/n] y

3) IBM Maximo Operator Catalog Selection
The IBM Maximo Operator Catalog is already installed in this cluster (v9-251127-amd64).  If you wish to install MAS using a newer version of the catalog please first update the catalog using mas update.

Catalog Details
Catalog Image:         icr.io/cpopen/ibm-maximo-operator-catalog:v9-251127-amd64
Catalog Digest:        sha256:3fce7a6fd2aaafd40597178512d7a4722f28252baec8ac5d4f41edafbd432d7d
MAS Releases:          9.2-feature, 9.1, 9.0, 8.11, 8.10
Cloud Pak for Data:    5.1.3
MongoDb:               8.0.13

What's New
MongoDb v8 support Running mas update will automatically upgrade existing MongoDbCommunity instances to MongoDb version 8. This upgrade is compatible with all MAS versions included in this catalog.
End of support for Openshift Container Platform 4.14 and 4.15, which are both now out of support (Since May 1 2025 and August 27 2025 respectively).
Security updates and bug fixes
 - IBM Maximo Application Suite Core Platform v8.10, v8.11, v9.0 and v9.1
 - IBM Maximo Manage v8.6, v8.7, v9.0 and v9.1
 - IBM Maximo IoT v8.7, v8.8, v9.0 and v9.1
 - IBM Maximo Monitor v8.10, v8.11, v9.0 and v9.1
 - IBM Maximo Optimizer v9.1
 - IBM Maximo Assist v9.1 and v9.0
 - IBM Maximo Visual Inspection v8.9, v9.0 and v9.1
 - IBM Maximo Real Estate and Facilities v9.1
 - IBM Maximo AI Service v9.1

Known Issues
- Customers using **Maximo Assist v8.7 or v8.8** should not update and must instead contact IBM Support for guidance regarding the removal of IBM Watson Discovery and upgrading to Maximo Assist v9.0
- If you have the Workday Connector installed and are upgrading from Manage 8.7.24 Multi-IS to Manage 9.0.17 (Online/Offline mode), it is advised to bypass this Patch 9.0.17 release due to a recently encountered error on build and await the subsequent patch release for 9.0.18. If you need to apply this patch, we can offer a Limited Availability Fix (LA Fix) in the event you encounter this issue. Please note that you will need to submit a LA Fix request for this.
- Customers using FIPS should not upgrade their MREF to 9.1.5, since it has problems during initialization.
- A known issue has been identified in Db2u warehouse operator in this catalog. Customers restoring/upgrading Db2, intermittently get db2 `SQL0290N  Table space access is not allowed.  SQLSTATE=55039` error causing connectivity issue between MAS and Db2. If you are facing this problem, please refer to the workaround provided in this [documentation](https://www.ibm.com/docs/en/cloud-paks/cp-data/5.3.x?topic=SSQNUZ_5.3.x/svc-db2/known-issues-dbs.htm#known-issues-dbs__db2-instance-fails-sql0290n__title__1).

Two types of release are available:
 - GA releases of Maximo Application Suite are supported under IBM's standard 3+1+3 support lifecycle policy.
 - 'Feature' releases allow early access to new features for evaluation in non-production environments and are only supported through to the next GA release.

┌────────────┬───────────────────────┬───────────────┬───────┬────────┬─────────┬─────────┐
│            │ 9.2-feature           │ 9.1-feature   │ 9.1   │ 9.0    │ 8.11    │ 8.10    │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Core       │ 9.2.0-pre.stable_4447 │ 9.0.17        │ 9.1.6 │ 9.0.17 │ 8.11.28 │ 8.10.31 │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Manage     │ 9.2.0-pre.stable_4817 │ 9.0.19        │ 9.1.6 │ 9.0.19 │ 8.7.26  │ 8.6.32  │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ IoT        │                       │ 9.0.14        │ 9.1.5 │ 9.0.14 │ 8.8.24  │ 8.7.28  │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Monitor    │                       │ 9.0.15        │ 9.1.5 │ 9.0.15 │ 8.11.23 │ 8.10.25 │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Assist     │                       │ 9.0.11        │ 9.1.5 │ 9.0.11 │ 8.8.7   │ 8.7.8   │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Optimizer  │ 9.2.0-pre.stable_3097 │ 9.0.16        │ 9.1.6 │ 9.0.16 │ 8.5.22  │ 8.4.23  │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Predict    │                       │ 9.0.10        │ 9.1.3 │ 9.0.10 │ 8.9.13  │ 8.8.11  │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Inspection │                       │ 9.0.14        │ 9.1.4 │ 9.0.14 │ 8.9.17  │ 8.8.4   │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ Facilities │                       │               │ 9.1.5 │        │         │         │
├────────────┼───────────────────────┼───────────────┼───────┼────────┼─────────┼─────────┤
│ AI Service │                       │               │ 9.1.9 │        │         │         │
└────────────┴───────────────────────┴───────────────┴───────┴────────┴─────────┴─────────┘
Select release 9.2-feature

4) License Terms
To continue with the installation, you must accept the license terms:
 - https://ibm.biz/MAS91-License
 - https://ibm.biz/MAXIT91-License
 - https://ibm.biz/MAXESRI91-License

Be aware, this channel subscription is supported for non-production use only.
It allows early access to new features for evaluation in non-production environments.
This subscription is offered alongside and in parallel with our normal maintained streams.
When using this subscription, IBM Support will only accept cases for the latest available bundle deployed in a non-production environment.
Severity must be either 3 or 4 and cases cannot be escalated.
Please refer to IBM documentation for more details.

Do you accept the license terms? [y/n] y

5) Configure Storage Class Usage
Maximo Application Suite and it's dependencies require storage classes that support ReadWriteOnce (RWO) and ReadWriteMany (RWX) access modes:
  - ReadWriteOnce volumes can be mounted as read-write by multiple pods on a single node.
  - ReadWriteMany volumes can be mounted as read-write by multiple pods across many nodes.

Storage provider auto-detected: OpenShift Container Storage (External)
  - Storage class (ReadWriteOnce): ocs-external-storagecluster-ceph-rbd
  - Storage class (ReadWriteMany): ocs-external-storagecluster-cephfs
Use the auto-detected storage classes? [y/n] y

6) Configure AppPoint Licensing
By default the MAS instance will be configured to use a cluster-shared License, this provides a shared pool of AppPoints available to all MAS instances on the cluster.

Alternatively you may choose to install using a dedicated license only available to this MAS instance.
  1. Install MAS with Cluster-Shared License (AppPoints)
  2. Install MAS with Dedicated License (AppPoints)
SLS Mode 1
SLS auto-detected: ibm-sls

Upload/Replace the license file? [y/n] n
Contact e-mail address test@test.com
Contact first name test
Contact last name test
IBM Data Reporter Operator (DRO) Namespace redhat-marketplace

7) Configure IBM Container Registry
IBM entitlement key *********

8) Configure MAS Instance
Instance ID restrictions:
 - Must be 3-12 characters long
 - Must only use lowercase letters, numbers, and hypen (-) symbol
 - Must start with a lowercase letter
 - Must end with a lowercase letter or a number
Instance ID testing

Workspace ID restrictions:
 - Must be 3-12 characters long
 - Must only use lowercase letters and numbers
 - Must start with a lowercase letter
Workspace ID test

Workspace display name restrictions:
 - Must be 3-300 characters long
Workspace name test

9) Configure Operational Mode
Maximo Application Suite can be installed in a non-production mode for internal development and testing, this setting cannot be changed after installation:
 - All applications, add-ons, and solutions have 0 (zero) installation AppPoints in non-production installations.
 - These specifications are also visible in the metrics that are shared with IBM and in the product UI.

  1. Production
  2. Non-Production
Operational Mode 2

10) Certificate Authority Trust
By default, Maximo Application Suite is configured to trust well-known certificate authoritories, you can disable this so that it will only trust the CAs that you explicitly define
Trust default CAs? [y/n] y

11) Cluster Ingress Secret Override
In most OpenShift clusters the installation is able to automatically locate the default ingress certificate, however in some configurations it is necessary to manually configure the name of the secret
Unless you see an error during the ocp-verify stage indicating that the secret can not be determined you do not need to set this and can leave the response empty
Cluster ingress certificate secret name

12) Configure Domain & Certificate Management
Configure domain & certificate management? [y/n] n

13) Single Sign-On (SSO)
Many aspects of Maximo Application Suite's Single Sign-On (SSO) can be customized:
 - Idle session automatic logout timer
 - Session, access token, and refresh token timeouts
 - Default identity provider (IDP), and seamless login
 - Brower cookie properties
Configure SSO properties? [y/n] n

14) Configure special characters for userID and username
By default Maximo Application Suite will not allow special characters in usernames and userIDs, and this is the recommended setting.  However, legacy Maximo products allowed this, so for maximum compatibilty when migrating from EAM 7 you can choose to enable this support.
Allow special characters for user IDs and usernames? [y/n] y

15) Adoption Metrics Reporting
Adoption Metrics are used by IBM to measure feature adoption, user engagement, and the success of product initiatives.
You can control three types of metrics:
 - Feature Adoption: Tracks feature usage to understand adoption and improve the product
 - Deployment Progression: Tracks progression of tasks and workflows within the product
 - Usability: Tracks user interface interactions to improve usability

When enabled (y), you permit IBM to capture and analyze these metrics to help improve the Maximo Application Suite experience.
When disabled (n), you are opting out of sending that specific metric type to IBM.
Enable feature adoption metrics? [y/n] y
Enable deployment progression metrics? [y/n] y
Enable usability metrics? [y/n] y

16) Enable Guided Tour
By default, Maximo Application Suite is configured with guided tour, you can disable this if it not required
Enable Guided Tour? [y/n] y

17) Application Selection
Install IoT? [y/n] n
Install Monitor? [y/n] n
Install Manage? [y/n] y
Install Assist? [y/n] n
Install Optimizer? [y/n] n
Install Visual Inspection? [y/n] n
Install Real Estate and Facilities? [y/n] y
Traceback (most recent call last):
  File "/opt/app-root/bin/mas-cli", line 62, in <module>
    app.install(argv[2:])
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 80, in wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 1949, in install
    self.interactiveMode(simplified=args.simplified, advanced=args.advanced)
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 80, in wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 1522, in interactiveMode
    self.configApps()
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 80, in wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 1099, in configApps
    self.configAppChannel("facilities")
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 80, in wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/install/app.py", line 1113, in configAppChannel
    versions = self.getCompatibleVersions(self.params["mas_channel"], appId)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/cli.py", line 98, in wrapper
    result = func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.12/site-packages/mas/cli/cli.py", line 291, in getCompatibleVersions
    return self.compatibilityMatrix[coreChannel][appId]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'facilities'

[Mahendran-R-000]

MREF 9.2 FC was released in March catalog : https://www.ibm.com/support/pages/node/7267277

However, the cluster is currently running version v9-251127-amd64, which does not include the required matrix for this release, causing the failure.

[vsolasa]

@Mahendran-R-000 Good observation however the newer catalog is also throwing the same error.

3. IBM Maximo Operator Catalog Selection

The IBM Maximo Operator Catalog is already installed in this cluster (v9-260326-amd64). If you wish to install MAS using a newer version of the catalog please first update the catalog using mas update.
Catalog Details
Catalog Image: icr.io/cpopen/ibm-maximo-operator-catalog:v9-260326-amd64
Catalog Digest: sha256:c0cc7bfca67a47d12a3fce6bcf6ad25ee83d75234c17b0e32e736af5e30b971c
MAS Releases: 9.2-feature, 9.1, 9.0, 8.11, 8.10
Cloud Pak for Data: 5.2.0
MongoDb: 8.0.17

[Mahendran-R-000]

@vsolasa Yes you are right. The Matrix is actually missing for facilities and the validation fails.

self.compatibilityMatrix: Dict[str, Dict[str, List[str]]] = {
            "9.2.x-feature": {
                "aibroker": ["9.2.x-feature", "9.1.x"],
                "manage": ["9.2.x-feature", "9.1.x"],
                "optimizer": ["9.2.x-feature", "9.1.x"],
                "visualinspection": ["9.2.x-feature", "9.1.x"],
            },

I tried by changing these things as per the other entries

sed -i '/"9.2.x-feature": {/,/}/ s/("visualinspection".*)/\1\n "facilities": ["9.2.x-feature", "9.1.x"],/'
/opt/app-root/lib64/python3.12/site-packages/mas/cli/cli.py

It is proceeding wit the next steps.

--manage-channel "9.2.x-feature"
--facilities-channel "9.2.x-feature" \

Not sure if other places needs some addition. FYI : I tried in [ibmmas/cli:19.7.0] and still we have the same issue. It needs to be addressed.

[vsolasa]

@anilprajapatiibm Could you please help assign this to someone at the earliest?

[terc1997]

hey, I can take this. Just to confirm, do we have Facilities in the feature channel ? Because I could be that it's missing in there intentionally.

[vsolasa]

Hi @terc1997 - Yes, here - https://www.ibm.com/support/pages/node/7267277

[terc1997]

@vsolasa I've reproduced the bug with the versions 19.6.0 and the latest released (19.8.0) and both showed the same problem. However, when I pulled the master code to work on the fix I saw that the code has been updated 4 days ago adding facilities in the matrix. See here. Therefore, we have two options:

• Recommend the customer to use the master version knowing that it's an unreleased version and some features can be flaky; or
• Verify if we can release a version in the following days and make it available to the customer.