Update to 3.2 without database-reset

[blackhawk60]

Hello,

we are running several SOPERA 3.1 infrastructure instances, both on Windows and Linux.

How can we safely update our installation to SOPERA 3.2 without loosing any data (e.g. user roles, policies, services, ...)?

Should any backup/restore-procedure be executed?

EDIT:
The "Release Notes for SOPERA ESB" (3.2) state that one has to export the LDAP data before uninstalling SOPERA3.1SP1.

Do we have to upgrade from 3.1 to 3.1SP1 first?

Are there any files within the SOPERA infrastructure's installation directory that have to be saved and restored? Examples are keystores, xml-files, ... .

Regards,

Marcus

[blackhawk60]

I've tried to backup the LDAP data as suggested by the release notes. However the restore did not work.

As OpenDS is the backend used by SOPERA, I followed [1] to backup all data. After installing SOPERA 3.2 I stopped all services and followed the restore procedure. In short, it did not work.

If I copy the old config directory into the new installation I am unable to restory any backend at all. I experimented with different sequences of restoring backends (each time starting from a vanilla SOPERA3.2 installation). All of them failed as shown below ("dry"==dry run, "real"==do actual restore):

-------------------
1. userRoot (real)
=> ok
2. config (real)
=> ok
3. schema (real)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/schema/
An error occurred while trying to load the Directory Server schema: An error
occurred while trying to decode the managed object configuration entry
cn=Numeric String Substring Matching Rule,cn=Matching Rules,cn=config: The
Substring Matching Rule could not be decoded due to the following reasons: The
"java-class" property must be specified as it is mandatory; The "enabled"
property must be specified as it is mandatory
(ServerManagementContext.java:786 ServerManagementContext.java:721
ServerManagementContext.java:466 ServerManagedObject.java:395
RootCfgDefn.java:3261 MatchingRuleConfigManager.java:133
SchemaConfigManager.java:163 DirectoryServer.java:2315 RestoreDB.java:375
TaskTool.java:493 RestoreDB.java:272 RestoreDB.java:133 RestoreDB.java:95)

--------------

1. schema (real)
=> ok
2. config (dry run)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/config/ -n
An error occurred while trying to load the Directory Server schema: An error
occurred at or near line 5 while trying to parse the configuration from LDIF
file /opt/SOPera-infra_Version32/openDS/config/config.ldif:
org.opends.server.util.LDIFException: Entry cn=config read from LDIF starting
at line 5 is not valid because it violates the server's schema configuration:
Entry cn=config violates the Directory Server schema configuration because it
includes attribute ds-cfg-entry-cache-preload which is not allowed by any of
the objectclasses defined in that entry
3. userRoot (dry run)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/userRoot/ -n
An error occurred while trying to load the Directory Server schema: An error
occurred at or near line 5 while trying to parse the configuration from LDIF
file /opt/SOPera-infra_Version32/openDS/config/config.ldif:
org.opends.server.util.LDIFException: Entry cn=config read from LDIF starting
at line 5 is not valid because it violates the server's schema configuration:
Entry cn=config violates the Directory Server schema configuration because it
includes attribute ds-cfg-entry-cache-preload which is not allowed by any of
the objectclasses defined in that entry

--------------

1. config (real)
=> ok
2. schema (dry)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/schema/ -n
An error occurred while trying to load the Directory Server schema: An error
occurred while trying to decode the managed object configuration entry
cn=Numeric String Substring Matching Rule,cn=Matching Rules,cn=config: The
Substring Matching Rule could not be decoded due to the following reasons: The
"java-class" property must be specified as it is mandatory; The "enabled"
property must be specified as it is mandatory
(ServerManagementContext.java:786 ServerManagementContext.java:721
ServerManagementContext.java:466 ServerManagedObject.java:395
RootCfgDefn.java:3261 MatchingRuleConfigManager.java:133
SchemaConfigManager.java:163 DirectoryServer.java:2315 RestoreDB.java:375
TaskTool.java:493 RestoreDB.java:272 RestoreDB.java:133 RestoreDB.java:95)
3. userRoot (dry)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/userRoot/ -n
An error occurred while trying to load the Directory Server schema: An error
occurred while trying to decode the managed object configuration entry
cn=Numeric String Substring Matching Rule,cn=Matching Rules,cn=config: The
Substring Matching Rule could not be decoded due to the following reasons: The
"java-class" property must be specified as it is mandatory; The "enabled"
property must be specified as it is mandatory
(ServerManagementContext.java:786 ServerManagementContext.java:721
ServerManagementContext.java:466 ServerManagedObject.java:395
RootCfgDefn.java:3261 MatchingRuleConfigManager.java:133
SchemaConfigManager.java:163 DirectoryServer.java:2315 RestoreDB.java:375
TaskTool.java:493 RestoreDB.java:272 RestoreDB.java:133 RestoreDB.java:95)

--------------

1. userRoot (real)
=> ok
2. schema (dry)
=> ok
3. config (dry)
=> ok
4. schema (real)
=> ok
5. config (dry)
=>
An error occurred while trying to load the Directory Server schema: An error
occurred at or near line 5 while trying to parse the configuration from LDIF
file /opt/SOPera-infra_Version32/openDS/config/config.ldif:
org.opends.server.util.LDIFException: Entry cn=config read from LDIF starting
at line 5 is not valid because it violates the server's schema configuration:
Entry cn=config violates the Directory Server schema configuration because it
includes attribute ds-cfg-entry-cache-preload which is not allowed by any of
the objectclasses defined in that entry

--------------

1. userRoot (real)
=> ok
2. config (real)
=> ok
3. schema (dry)
=>
root@sopera4:/opt/SOPera-infra/openDS/bin# ./restore -d /tmp/sopera4-ldap-backup/by-backup-tool/schema/ -n
An error occurred while trying to load the Directory Server schema: An error
occurred while trying to decode the managed object configuration entry
cn=Numeric String Substring Matching Rule,cn=Matching Rules,cn=config: The
Substring Matching Rule could not be decoded due to the following reasons: The
"java-class" property must be specified as it is mandatory; The "enabled"
property must be specified as it is mandatory
(ServerManagementContext.java:786 ServerManagementContext.java:721
ServerManagementContext.java:466 ServerManagedObject.java:395
RootCfgDefn.java:3261 MatchingRuleConfigManager.java:133
SchemaConfigManager.java:163 DirectoryServer.java:2315 RestoreDB.java:375
TaskTool.java:493 RestoreDB.java:272 RestoreDB.java:133 RestoreDB.java:95)
--------------

Was anyone successful in upgrading from SOPERA 3.1 to SOPERA 3.2 without resetting the entire LDAP content? Any help/ideas are appreciated!

Regards,

Marcus

[1] https://www.opends.org/1.0/page/HowToBackUpAndRestoreData#section-HowToBackUpAndRestoreData-ToRestoreForDisasterRecovery

[blackhawk60]

One (hopefully final) addition from my side.

Since the restore/backup tools do not work, I tried to use ldif-export/-import. It didn't work either. I was able to export and import the userRoot backend, my services even showed up when launching the AdminTool. However when I run my provider, error messages come up.

Regards,
Marcus

[Michael]

Hi Marcus,

the internal structures of the backends have changed from version 3.1 (base) to version 3.2. This is the reason why your attempt with LDIF export/import doesn't work.

With AdminTool Commandline or the AdminTool Facade it is possible to get all Service-Artefakts and configurations script based from 3.1 infrastructure deploy them again into the 3.2 Infrastructure.

Infos about AdminTool Commandline: http://www.sopera.de/SOP-Library/ASF_32/Operations_and_Administration/wwhelp/wwhimpl/js/html/wwhelp.htm#href=Operations_and_Administration/oap_html.html in Chapter "Administration Guide - Command-Line Interface"

Infos about AdminTool Facade http://www.sopera.de/SOP-Library/ASF_32/AdminTool-FACADE/index.html

Hope this helps :)

[blackhawk60]

Hi Michael,

thanks for the reply!

As I read it, I have to execute the following commands to export everything:

run-admin-tool.sh -pwd ... -usr ... -m export -s security -p <dst-filename>

run-admin-tool.sh -pwd ... -usr ... -m export -s serviceRegistry -p <dst-filename>

run-admin-tool.sh -pwd ... -usr ... -m export -s configurationTree -tree SOP -p <dst-filename>

run-admin-tool.sh -pwd ... -usr ... -m export -s configurationTree -tree SBB -p <dst-filename>

Are these commands sufficient? Is there any data they do not cover?

I did so in the past and was unable to export passwords. Can they be extracted as well?

You said the internal structure changed considerably. Just to make sure: Is the data exported from 3.1 using the commands above compatible to a 3.2 installation?

Thanks!

Marcus

[afuchs]

Marcus,

please follow the following procedure to import your data into SOPERA 3.2:

1. Export Service Registry Data:

run-admin-tool.sh -pwd ... -usr ... -m export -s serviceRegistry -p <dst-filename>

2. Export Configuration Tree Identifier "SBB":

run-admin-tool.sh -pwd ... -usr ... -m export -s configurationTree -tree SBB -p <dst-filename>

Note:
Configuration changes made within the tree identifier "SOP" cannot be exported/imported in an automated fashion due to modifications in data structures in SOPERA 3.2. Fortunately, the SOP-tree does not contain any mass data nor does it have many parameters customers need to change frequently.

3. Export of authentication/authorization data:
This can be done most easily by means of an LDAP-Browser, apache directory studio is a good-one:

http://directory.apache.org/studio/downloads.html)

3.1 Necessary data for your connection to the SOPERA LDAP:

Host: Hostname of your SOPERA infrastructure server
Port: LDAP-port of your LDAP-server, default: 10389
Bind-DN: cn=admin,o=SOPware
password: secret

3.2 Export:

Export the trees o=SOPware,ou=DataAuthenticationTSP and
o=SOPware,ou=DataAuthorizationTSP
as LDIF-files.

4. Import of Service Registry and Configuration Data in SOPERA 3.2

Import the data exported in (1) into your SOPERA 3.2 environment using the SOPERA Admin-Tool:

4.1 Service Registry:

SOPERA Admin-Perspective=>Service Registry-View
Right-click=>Publish=>Browse=>FileSystem got to your export-folder
do NOT select “use SOPERA folder structure” =>OK.

4.2 Configuration data:

SOPERA Admin-Perspective=>Configuration-View
right-click on Tree-Identifier “SBB”=>Import=>complete Tree=>Go to the XML-file created in (1),
confirm „overwrite inherited configuration“ and press „OK“.

5. Import of authentication/authorization data:

5.1 Configure your OpenDS LDAP-server:

Edit the file <OPENDS_HOME>/config/config.ldif

Replace the line ds-cfg-allow-pre-encoded-passwords: false
with
ds-cfg-allow-pre-encoded-passwords: true

Restart OpenDS.

5.2 Import the .ldif files:

-Connect to your SOPERA 3.2 LDAP as described in (3).
-Delete the trees:
o=SOPware,ou=DataAuthenticationTSP and
o=SOPware,ou=DataAuthorizationTSP

-Import of .ldif-files: right-click on o=SOPware=>import=>import LDIF

Using this procedure, the complete security data including encrypted passwords are in your new SOPERA 3.2 environment.

Cheers,
Alexander