IBMi-bob migration log

[cosentino-smeup]

I am migrating a library with IBMi-bob. I am using this documentation.

I tried with many SRC-PF but the migration always fails. There is just 1 error with code "1":

/tmp/vscodetemp-O_svPnzGtK/QILEGEN: export PATH="/QOpenSys/pkgs/bin:$PATH:" && /QOpenSys/pkgs/bin/makei cvtsrcpf -c *JOB -t -l QILEGEN P_FENS
{
    "code": 1,
    "signal": null,
    "stdout": "",
    "stderr": "bsh: PATH=/QOpenSys/pkgs/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin:: is not an identifier"
}

There is also another thing I don't understand: before to start the migration, I executed the following command to set the current path
cd /SMEDOC/BOB/P_FENS which is my destination folder

nevertheless in some messages I can see the path to my home folder:
/home/COSANT

For example:

/home/COSANT: mkdir -p /tmp/vscodetemp-O_svPnzGtK/SRCWK
{
    "code": 0,
    "signal": null,
    "stdout": "",
    "stderr": ""
}

Could you please help me to understand what is the problem?

I attached the log file.

migration.log

[SanjulaGanepola]

Hello @cosentino-smeup.

Looks like the issue is because of the extra colons :: in the PATH variable. Since the PATH variable should already be set properly, I updated our extension to avoid setting it. I just pushed an update so can you update the Project Explorer extension to 2.11.1 and have a try again?

There is also another thing I don't understand: before to start the migration, I executed the following command to set the current path
cd /SMEDOC/BOB/P_FENS which is my destination folder

nevertheless in some messages I can see the path to my home folder:
/home/COSANT

The path which you see here in the log is simply where the command is being run in on the IBM i. This is controlled by the extension. In most cases, the user's home directory will be used unless the command needs to run in a specific directory. Could you clarify what you mean by /SMEDOC/BOB/P_FENS is your "destination folder".

[cosentino-smeup]

Hello @cosentino-smeup.

Looks like the issue is because of the extra colons :: in the PATH variable. Since the PATH variable should already be set properly, I updated our extension to avoid setting it. I just pushed an update so can you update the Project Explorer extension to 2.11.1 and have a try again?

There is also another thing I don't understand: before to start the migration, I executed the following command to set the current path
cd /SMEDOC/BOB/P_FENS which is my destination folder
nevertheless in some messages I can see the path to my home folder:
/home/COSANT

The path which you see here in the log is simply where the command is being run in on the IBM i. This is controlled by the extension. In most cases, the user's home directory will be used unless the command needs to run in a specific directory. Could you clarify what you mean by /SMEDOC/BOB/P_FENS is your "destination folder".

Thanks @SanjulaGanepola for your fix. I will download and test it straightaway and I will give you a feedback as soon as I completed the tests.

This is my first migration with IBMi-bob, so maybe I didn't get everything or misunderstood something even if I have read the documentation.

That said, what I meant by "destination folder" is that I expect the migration will convert the source from the library I have chosen from the object browser and put the new source files in the "destination" folder "/SMEDOC/BOB/P_FENS" on IFS. To set the folder "/SMEDOC/BOB/P_FENS" as current directory, I used the PASE terminal and executed the following command:

cd /SMEDOC/BOB/P_FENS

In fact, this document says:

"Converts all members in a source physical file to properly-named (Bob-compatible), UTF-8 encoded, LF-terminated source files in the current directory in the IFS. Generally speaking, the source member type will become the filename extension."

[cosentino-smeup]

I installed the version 2.11.1 of the library "Project Explorer extension". I don't get the previous error message anymore but another one comes out:

/tmp/vscodetemp-O_eAQjPLaL/£UI_SRC: /QOpenSys/pkgs/bin/makei cvtsrcpf -c *JOB -t -l £UI_SRC P_FENS
{
    "code": 1,
    "signal": null,
    "stdout": "It looks like /QOpenSys/pkgs/bin/ is not currently in your system PATH. \nThis means that programs and tools installed in this directory may not be \neasily accessible from the command line.\n\nTo add /QOpenSys/pkgs/bin/ to your system PATH (if it is not already present), \nuse the following command:\n\nexport PATH=$PATH:/QOpenSys/pkgs/bin\n\nYou can also add this command to your shell profile file \n(e.g. ~/.bashrc or ~/.bash_profile) to make the change permanent.\n\npython3 is not installed or not in your system PATH.",
    "stderr": ""
}

[cosentino-smeup]

Please find attached the migration log.

migration-2.log

[cosentino-smeup]

Hello @SanjulaGanepola, I was able to migrate some sources after I created the following files in my home folder:

/home/COSANT/.bashrc:
export PATH=/QOpenSys/pkgs/bin:$PATH

/home/COSANT/.profile:

PATH=/QOpenSys/pkgs/bin:$PATH
export PATH PASE_PATH

[SanjulaGanepola]

That is great to hear. Yes, a .bashrc file is required to be created in your home directory for all this to work. In fact, the Code for IBM i extension should provide you the following prompt to have one created for you upon first connecting:

After clicking Yes, it will create one similar to this:

# Generated by Code for IBM i
export PATH=/QOpenSys/pkgs/bin:$PATH:/QOpenSys/usr/bin:/usr/bin

In some cases, due to caching, you may not see this prompt even though you do not have a .bashrc file (which is what happened for you). In this case, you can right-click on the connection and use the Connect and Reload Server Settings option to trigger the check for if it exists.

This is my first migration with IBMi-bob, so maybe I didn't get everything or misunderstood something even if I have read the documentation.

No worries, great to hear you are using this tool and happy to help 😄

That said, what I meant by "destination folder" is that I expect the migration will convert the source from the library I have chosen from the object browser and put the new source files in the "destination" folder "/SMEDOC/BOB/P_FENS" on IFS.

As for this issue, the main purpose of this migration tool is to migrate source to your local workspace for local development. So while Project Explorer does use cvtsrcpf to migrate source first to the IFS and then to the local workspace, it will use a temporary IFS directory to convert the files and they get deleted at the end of the process. This is why in the log you saw the working directory to be /tmp/vscodetemp-O_svPnzGtK when the cvtsrcpf command was being run.

Is your goal to migrate source to the IFS or the local workspace? If you would like the source to also be in the IFS, after migrating the source to your local workspace, you can set your deploy directory and then deploy your entire project there. This is explained here

To set the folder "/SMEDOC/BOB/P_FENS" as current directory, I used the PASE terminal and executed the following command:

cd /SMEDOC/BOB/P_FENS

Changing directory in the PASE terminal will only apply for the terminal session which you are in. As mentioned previously, the current directory in which the commands are run in are controlled by the extension.

[cosentino-smeup]

Thank you very much @SanjulaGanepola for the explanation.

Is your goal to migrate source to the IFS or the local workspace? If you would like the source to also be in the IFS, after migrating the source to your local workspace, you can set your deploy directory and then deploy your entire project there. This is explained here

My goal in this migration phase is:

• migrate my IBMi library to my local workspace
• when everything will be migrated, I will push to github
• from github I want to create a github action which will copy the sources files to the IFS folder

When I finished the migration, the workflow will be approximately the same:

• make the changes to my local workspace
• push to github in a branch
• from github I want to create a github action which will intercept the approval of the pull request and will copy the sources files to the IFS folder
• then, I want to compile the code

So, according with what you said before, I think I don't need to use the deploy action from the project explorer, because I want Github to push the changes into IFS automatically.

At the moment I still can't migrate some source files and I don't understand why. For example, in the file SRCDZ, there are 4 PF sources files. Only 3 over 4 get migrated. One of them gives a error, but I can't understand what is the problem.

/tmp/vscodetemp-O_VRHf9Ulw/SRCDZ: /QOpenSys/pkgs/bin/makei cvtsrcpf -c 00280 -t -l SRCDZ P_FENS
{
    "code": 1,
    "signal": null,
    "stdout": "4 source members found.\nConverting $DIZ_ED to $diz_ed.pf\n>  CPYTOSTMF FROMMBR('/QSYS.LIB/P_FENS.LIB/SRCDZ.FILE/$DIZ_ED.MBR') TOSTMF('/tmp/vscodetemp-O_VRHf9Ulw/SRCDZ/$diz_ed.pf') ENDLINFMT(*LF) STMFCCSID(1208) STMFOPT(*REPLACE)\nSuccessfully imported member text!\nConverting $DIZ_JA to $diz_ja.pf\n>  CPYTOSTMF FROMMBR('/QSYS.LIB/P_FENS.LIB/SRCDZ.FILE/$DIZ_JA.MBR') TOSTMF('/tmp/vscodetemp-O_VRHf9Ulw/SRCDZ/$diz_ja.pf') ENDLINFMT(*LF) STMFCCSID(1208) STMFOPT(*REPLACE)\nSuccessfully imported member text!\nConverting $DIZ_C£ to $diz_c£.pf\n>  CPYTOSTMF FROMMBR('/QSYS.LIB/P_FENS.LIB/SRCDZ.FILE/$DIZ_C£.MBR') TOSTMF('/tmp/vscodetemp-O_VRHf9Ulw/SRCDZ/$diz_c£.pf') ENDLINFMT(*LF) STMFCCSID(1208) STMFOPT(*REPLACE)",
    "stderr": "Traceback (most recent call last):\n  File \"/QOpenSys/pkgs/lib/bob/src/makei/cli/makei_entry.py\", line 293, in <module>\n    cli()\n  File \"/QOpenSys/pkgs/lib/bob/src/makei/cli/makei_entry.py\", line 41, in cli\n    args.handle(args)\n  File \"/QOpenSys/pkgs/lib/bob/src/makei/cli/makei_entry.py\", line 269, in handle_cvtsrcpf\n    CvtSrcPf(args.file, args.library, args.tolower, args.ccsid, args.text).run()\n  File \"/QOpenSys/pkgs/lib/bob/src/makei/cvtsrcpf.py\", line 116, in run\n    member_text = result[0][0][0]\nTypeError: 'NoneType' object is not subscriptable"
}

Note: the source $DIZ_ED has been migrated.

Is there a way to get a detailed log with the error descriptions in order to understand what the problems are?

[cosentino-smeup]

Also, I noticed that the migration didn't update the /COPY path.

Is there a way to do it automatically?

In my iproj.json I have set the "includePath" property

{
  "version": "0.0.1",
  "description": "p-fens for IBM BOB test",
  "objlib": "P_FENC",
  "curlib": "P_FENC",
  "includePath": [
    "includes",
    "QILEGEN",
    "QPROGEN",
    "QAPIGEN"
  ],
  "preUsrlibl": [
    "P_FENC"
  ],
  "postUsrlibl": [],
  "setIBMiEnvCmd": [],
  "repository": "https://github.com/smeup/p-fens.git",
  "compileCommand": "/QOpenSys/pkgs/bin/makei c -f {filename}",
  "buildCommand": "/QOpenSys/pkgs/bin/makei build"
}

$b£g02b.rpgle.txt

[cosentino-smeup]

Hello @SanjulaGanepola I think I migrated everything. The problem was a parameter that changed the name of the source files to lowercase.

[SanjulaGanepola]

Glad it is working now!

So, according with what you said before, I think I don't need to use the deploy action from the project explorer, because I want Github to push the changes into IFS automatically.

Based on what you described, you are right in that deploy from Project Explorer will not be needed.

• from github I want to create a github action which will intercept the approval of the pull request and will copy the sources files to the IFS folder
• then, I want to compile the code

In case you have not heard of @ibm/ibmi-ci, you should definitely check it out as it can be used for what you are trying to achieve here. Example of it being used in a GitHub action here as well.

Also, I noticed that the migration didn't update the /COPY path.

Is there a way to do it automatically?

At the moment, the current include/copy directive fixing involves updating the source to use Unix style paths instead of member styled paths. I believe there is no support as of right now for checking the includePath in iproj.json and updating it automatically. I will mention this as an enhancement for source orbit which is what Project Explorer uses for this.

[cosentino-smeup]

In case you have not heard of @ibm/ibmi-ci, you should definitely check it out as it can be used for what you are trying to achieve here. Example of it being used in a GitHub action here as well.

Thank you @SanjulaGanepola, that's a great advise. I will definitely check it out!

At the moment, the current include/copy directive fixing involves updating the source to use Unix style paths instead of member styled paths. I believe there is no support as of right now for checking the includePath in iproj.json and updating it automatically. I will mention this as an enhancement for source orbit which is what Project Explorer uses for this.

What are my options to bypass the problem in the mean time that SourceOrbit considers whether to implement this feature or not?

Do I have to modify all manually?

[SanjulaGanepola]

I brought up the idea to @worksofliam who manages source orbit and he agreed it would be a great feature. I have opened an issue to track the enhancement here.

Until it is added, the only current option would be to do this manually. However, adding this feature will probably be looked at next week so hopefully it should be out soon.

[cosentino-smeup]

I brought up the idea to @worksofliam who manages source orbit and he agreed it would be a great feature. I have opened an issue to track the enhancement here.

Until it is added, the only current option would be to do this manually. However, adding this feature will probably be looked at next week so hopefully it should be out soon.

Thanks @SanjulaGanepola that will be a great improvement !!! I'll wait.

[cosentino-smeup]

In case you have not heard of @ibm/ibmi-ci, you should definitely check it out as it can be used for what you are trying to achieve here. Example of it being used in a GitHub action here as well.

Thank you @SanjulaGanepola, that's a great advise. I will definitely check it out!

At the moment, the current include/copy directive fixing involves updating the source to use Unix style paths instead of member styled paths. I believe there is no support as of right now for checking the includePath in iproj.json and updating it automatically. I will mention this as an enhancement for source orbit which is what Project Explorer uses for this.

What are my options to bypass the problem in the mean time that SourceOrbit considers whether to implement this feature or not?

Do I have to modify all manually?

Hello @SanjulaGanepola, I was checking out the documentation of ibm-ci but I don't understand if it must be installed on IBMi, the client or Github. Would you please suggest me a documentation about the procedure to install it?

Thank you in advance for the advice

[SanjulaGanepola]

@cosentino-smeup ibmi-ci does not get installed on the IBM i. It can be installed on your local machine using the command npm i -g @ibm/ibmi-ci and then you can use ici to see the parameters which can be used. If you plan to use it locally, you will need to setup the environment variables as it suggests.

The main use case for this tool will however be in a GitHub action. In this case, you can create a yaml file with a step for the installation like this and then execute a deploy/build like this. Similar to running locally, note that environment variables will need to be set like this meaning that you will need to setup up GitHub repository secrets to store credentials to your IBM i.

You can also checkout Liam's blog post on this tool here for more info.

[edmundreinhardt]

FYI @cosentino-smeup
The happy path for migration is to

1. Migrate to local workspace vis the tools provided
2. Publish to github at your leisure, test and official build pipelines will triggered from this source of truth
3. But for active development you deploy to a personal IFS directory and build into a personal library and test there before comitting to git and trigger ci/cd from there