Add API call to build a json message, given a file in the proper tree.

[petersilva]

Add a properties to the __init__(), for missing elements. and an API call fileToMsg( relPath )
that builds a sample message with the appropriate topics.

[petersilva]

ok, there is an initial working version on the issue018 branch. It can be tested by:

cd GTStoWIS2
python3 __init__.py

it will create the content for a bunch of messages, for files which have been added to the sample_GTS_data folder
It executes, but have not had time to validate that the checksums produced are correct.

sample output:

path: SZAU01_AMMC_111809_c87a603174d55735bb57d91621db1751.txt
message is: {"baseUrl": "file://", "relPath": "au/melbourne_world_met_centre/surface/sea/tsunami/au/SZAU01_AMMC_111809_c87a603174d55735bb57d91621db1751.txt", "pubTime": "20210111T184306.794926882", "integrity": {"method": "sha512", "value": "zzVurIaqW0nqWdid1jGj6YKS1MYuJRsAEr/Qte30lYbquESK7P9fLkFkUsix1nMmqAMoIRxW09TWpKmO6nonSw=="}, "mtime": "20210111T180949.0906932354", "atime": "20210111T181529.726648331", "mode": "664"}

@josusky ? maybe you can grab the sample data from the repo, calculate the checksums and compare to the output of this thing?

[petersilva]

OK added content embedding. made one sample file smaller so it could be included.

[josusky]

I had to add "retPath" pointing to the real location of the file. The message/notification with URL only (no content) has correct hash, i.e. ended as "Input OK [verified]", but the embedded message failed. I am going to check it.

[josusky]

Err, one letter variables ("m", "h" ...), @petersilva , your code would not get through me into production :-)

[josusky]

OK, the notification with embedded content/message is OK too. I have unintentionally corrupted it (added new lines) by copying the script output through clipboard in a lame way, sorry.
Still, think about improving readability of your code :-)

[petersilva]

My tendency is to use 1 letter names when it is a routine that is a few lines long and the variable is core of what the routine is doing, or it is a short-use temporary thing... so it is really obvious (in my view) from context. To me, shorter names are good in code that is simple. I agree that in convoluted or complex processing too many one letter variables are a problem, but one or two at a time in a routine where it's obvious seems practical to me. But If you have a style-guide or other guidance to suggest, that's fine.

I'm not sure if you are complaining about 1 letter variables in the entire module (there are a lot of those.) We can use this issue as a template to adjust the style. so for this issue I imported the time conversion routines, and added the mapAHLtoMessage() entry point. I guess we can talk style in relation to that.

One letter variables in mapAHLtoMessage:

m - the python dictionary holding the properties of the message that will be returned at the end of the routine. It's on almost every line of code in the routine.
h - for hash... the checksum data structure, used in three lines to calculate a checksum.

naming them "messageToReturn" , or "hashToCalculate" does not add clarity in my view. It should be obvious ... I mean the

The description of the routine says that it is building a python dictionary, the first line of code is

   m={}

and the last line of code is

return m

and it's the only exit path.

dunno... open to advice.

[petersilva]

oh... I added comments describing the time routines... which also had 1 letter variables... perhaps that helps?

[petersilva]

about the retPath. I'm glad you did that. makes it complete as-is. Another option I was considering was to copy the file into the appropriate sub-dir, so that relPath works. What do you think is better for demonstration purposes?

[josusky]

@petersilva when it concerns the one letter variables, I have jumped into the middle of the function by searching for "content" and got confused for a while. Sure, I was able to understand it, after scanning back and reading the whole thing. And I agree that "messageToReturn" would be unnecessary long in this context. I would be happy with "message".
Other style issue is inconsistency in the use of white spaces, one hand there is:
m[ 'integrity' ] =
on the other
m['mtime']
Other example
m = {} vs h=sha512()
This is all nitpicking, of course. But ... an experienced developer, like you, should write code that pleases the eye of the reader :-)

[josusky]

When it concerns the "relPath" vs "retPath", as the difference popped up by itself, perhaps we could use this example also to explain/demonstrate it too.

[josusky]

@petersilva pushed "retPath" changes to branch "issue018" for your review.

[petersilva]

Your changes look good to me. In addition,
I pushed:

• a change of the name to msg
• ran the whole thing through yapf3 to get PEP8 compliance.

I don't mind the nitpicking at all, really do want it to be pretty so people like what they see. But of course one cannot please everyone, so discussion is needed to clarify. In the Canadian stack, I started using yapf3 as a pre-processor on all commits to ensure consistency with pythonic style. It should help with inconsistencies that annoy experienced devs.

If that looks better, we can do the same?

[petersilva]

oh... I just noticed... basePath... I'd prefer if that were a property... I pushed that...

[josusky]

I like the consistency created by "yapf3". As you said, different people may prefer different styles. Over time I learned to adjust, so if you like e.g. space around bracket, I can accept it - as long as it is applied consistently. I guess, that many people have it like me, that is the reason why automatic formatters like "yapf3" and "black" are getting popular :-)

[josusky]

When it concern the new property "basePath", I am a bit confused. Isn't it logically included in the "baseUrl"? Actually, I have tried to change it and it did not change the result.

[petersilva]

example:

• If you are serving http:
• documentRoot (the directory on the http server that is the top of the web-tree) /var/apache/html
• you might have a sub-directory under there, called WIS to serve the tree being replicated.
  so:

baseURL=https://myserver/WIS
basePath=/var/apache/html/WIS

If you serve the same location with a file url... it is:

baseURL=file:/var/apache/html/WIS.

I would expect the basePath to be a constant value for everything posted, just like a baseURL is, which
is why I thought it made more sense as a property, rather than something supplied with each file.

[petersilva]

as per the previous note, added f9e9085 to issue018 branch. It makes adding the basePath to the baseURL conditional on using the "file:* scheme. For other protocols, one would not do this.

[petersilva]

added some more samples, so in some cases there is no retPath ( 400eda9 )

[petersilva]

no mandate for payload format. abandoning generation of reference payload.