manual encoder support

[phil-barrett]

Since this was brought up in a different thread, I thought it a good idea to open a separate issue. Hopefully it will attract a bit more input here.

The encoder support currently works pretty well but there is the question of how big an effect each "pulse" has. Quoting Terje:

Currently there is no config setting for encoder sensitivity, that is how big the adjustment will be for one turn on the encoder. This will likely be added in the next release - I am thinking of adding settings for a number of encoders, perhaps up to 5?

I assume that shaft encoder support would be in a different bit of code and this is only about manual encoders - aka jog or speed dial.

5 is probably more than enough to capture the range of needs.

There seem to be 2 common hand turned (ie not motor/shaft) encoder types readily available from amazon/ebay/ali/... - 20 (and 24) ppr and 100 ppr. The 20/24 type is the "arduino" one and the 100 type is the larger kind you see on a lot of MPGs. The 100 ppr kind seems to be an MPG de facto standard. A look through digikey/mouser shows there are a lot more possibilities but most are shaft encoders. There are quite a few 12 ppr encoders that are very low cost but I don't see them very often on the popular sites.

For basic support, does it make sense to have the effect of a full turn of an encoder be approximately the same independent of PPR? Worth a little experimentation to get a better sense of it.

Probably not coming soon but a very nice feature would be an adaptive algorithm that changes the movement step based on speed of encoder rotation. Turning it very slowly would result in very small changes, fast results in larger changes.

[terjeio]

I assume that shaft encoder support would be in a different bit of code and this is only about manual encoders - aka jog or speed dial.

Correct, there is a setting already defined for PPR (Pulses Per Revolution) for the spindle encoder.

For basic support, does it make sense to have the effect of a full turn of an encoder be approximately the same independent of PPR?

That is where the sensitivity (or whatever it should be called) comes into play.

Turning it very slowly would result in very small changes, fast results in larger changes.

I've kind of done that in my MPG & DRO implementation, however it could be improved upon. Turning speed of the MPG encoders affects the feed rate.

[phil-barrett]

That's Terje, always one step ahead...

[terjeio]

I have added support for up to 5 encoders with a number of settings - 4 defined this far, up to ten available.

Encoder mode:

0. Universal - single click toggles between feed rate, rapid rate and spindle RPM override, double click resets to 100%. Defaults to feed rate override on startup. A message is sent on mode change. A possible extension could be to add a real time report on a mode change (or report all request) for senders to reliably highlight the active override, e.g. by changing the background colour of the override percentage field.

1. Feed rate override - single or double click resets to 100%

2. Rapid rate override - single or double click resets to 100%

3. Spindle RPM override - single or double click resets to 100%

Further modes for MPG input defined but not yet supported by the plugin.

Encoder pulses per revolution:

Not yet clear to me how to handle this in a proper way. A different/additional setting might be required.

Encoder pulses per detent:

Is often four.

Is needed for the rapid rate override mode as there are only three steps available. Impossible to set unless the pulse count is divided down. The other option is to always divide the pulse count by number of detents for other modes as well.

Double click sensitivity:

100 - 900 ms, defaults to 500 ms.

I will commit these changes to the test branch later.

EDIT: spelling

[phil-barrett]

Looks good. I think some sort of visual indication of which override is active is very desirable.

There are encoders with no detents. Personally I prefer detents since they give haptic feedback.

PPR setting/usage. There is a bit of ambiguity here. What the industry calls PPR is 1/4 the number of counts (state changes) that grblHAL sees (due to grey code). So a 100 PPR produces 400 counts per rev. How are you handling the encoders on your booster board? Those appear to be 100 ppr. You mentioned an adaptive approach. The way it is currently set up in the current github distribution a 100 PPR encoder maxes out on spindle override at 1/4 of a rev. Maybe an encoder profile that can be selected would make sense. It would have PPR, detents/rev, sensitivity (steps per full state change or rev or detent), acceleration.

Obviously acceleration needs an understandable way to express it. Maybe something related to inverse steps per second squared. I guess that would be seconds squared per step. I will play with this in my MPG. It is especially useful in moving an axis. Right now many (most?) MPGs have a sensitivity setting (mine has 1X,10X,100X) so maybe a simpler approach is to just map time between pulses to select a feed rate or distance with the jump points (in seconds between pulses) selectable in the profile.

[terjeio]

Personally I prefer detents since they give haptic feedback.

For menu navigation and overrides yes, for MPG control no so sure. I have problems with jerky movements due to detents after I increased the acceleration and max rate settings. Hopefully this can be fixed in code.

The way it is currently set up in the current github distribution a 100 PPR encoder maxes out on spindle override at 1/4 of a rev.

The new settings should take care of that. In the new version I scale the actual count to 100 counts per revolution, hopefully this is acceptable. If not an additional setting is required. Be aware that override commands are buffered and acted upon by the grbl foreground process. The buffer size is currently 16 deep, perhaps this should be doubled or quadrupled in order to avoid losing commands. Buffer size is set here:

grblHAL/grbl/override.h

Line 27 in 0c0d6af

#define OVERRIDE_BUFSIZE 16 // must be a power of 2

Obviously acceleration needs an understandable way to express it.

Not sure what you mean by this - it is not possible to change the axis acceleration on the fly. In my MPG implementation I get velocity from QEI peripheral in the processor, currently I multiply this with 50 to set the feed rate for the G1 commands I send. IIRC the function to create the G1 commands get called every 200 ms.

In my current implementation one revolution equals 1mm at 1X, 10X is usable but 100X not so I dropped that.

I read an article some time ago about making electronic handwheels behaving in a similar way as their mechanical counterparts, even with haptic feedback from the machine? I'll see if I can locate it again as it might provide useful information.

[phil-barrett]

By acceleration I mean changing the multiplier based on time between pulses.

[HuubBuis]

Probably not coming soon but a very nice feature would be an adaptive algorithm that changes the movement step based on speed of encoder rotation. Turning it very slowly would result in very small changes, fast results in larger changes.

I have used this algorithm some years ago as I made my first stepper driver/controller board for an Atmega32 processor, written in c++ and a low resolution rotary encoder (knob) . If you want, I can dig up the code, must be archived some where.
The same code is also used in my power supply to set the output voltage and max current.

[terjeio]

an adaptive algorithm that changes the movement step based on speed of encoder rotation.

I guess I have to read more carefully. In my MPG implementation the movement step is based on the count not velocity. The feed rate is controlled by the speed of encoder rotation (velocity). You want the target position to change with velocity if I understand you correctly. The count is then only used to calculate velocity?

So there are two possible modes then:

• movement step based on count (position) and feed rate based on velocity. This is mechanical handwheel emulation. I send relative or absolute G1 commands to the controller based on the current distance mode. In my MPG I change the colour of the axis DRO while the encoder is turned and display the target position instead of the position read from the controller.

• movement step based on velocity. For jogging with fixed feedrate? Or both target position and feed rate based on velocity?

If you want, I can dig up the code, must be archived some where.

Would be great if you can do so.

@jschoch has done some work on this as well, but AFAIK has not published any code yet. He used a second STM32 BluePill to handle the encoder input.

[jschoch]

I think the version from that video is here https://github.com/jschoch/grbl-MITM-pendant/tree/grblHAL_testing

It is quite a mess because it was derived from a different project where I was experimenting with having an MPG for all 3 axis. It has output for a OLED screen which also jumbles up the code.

The way it works is that is uses a 2nd UART to send gcode commands to grblHAL. You need to reconfigure your grblHAL to map the 2nd UART and map a button to toggle this input 2nd mode.

The encoder has 2400/rev resolution and no detents. You'd want to change the parameters quite a bit to tune it for a 100/rev encoder.

There is an ISR that gets updated when the encoder pulses. This also calculates the velocity. There are multiple modes and a mode button. It has a simple state machine to figure out what to do. The "acceleration mode" attempts to only use the change in acceleration to determine the jog distance. The "inc mode" will adjust the feedrate based on the velocity but keep the distance fixed. It also monitors the receive buffer of grblHAL to prevent filling up the buffer. If the motion stops a jog cancel is issued.

the tricky part is to ensure you can stop with as little lag as possible while keeping the jog distance and feedrate as high as possible.

formula:
  
  dt is estimated jog time
  v current jog rate
  N is the number of planner blocks N=15
  dt > v^2 / (2 * a * (N-1))
  T = dt * N  ; computes latency
  max_jog_rate is rate in mm/s/s
  a is the max acceleration along the jog vector
  estimated_jog_time = current_jog_rate / (2 * max_jog_rate_accel *10)
  jog_distance = current_jog_rate * estimated_jog_time
  

I'm happy to answer any questions about it.

[HuubBuis]

Would be great if you can do so.

I have found the code I use to process encoder pulses and made an Arduino sketch. The code is documented in Dutch, tomorrow I will translate it to English and post this code.

If the motion stops a jog cancel is issued
I also issue a jog cancel (in my gui) when the direction changes and I use relative jog moves what makes the timing less critical.

[HuubBuis]

This is an Arduino program that shows how i process rotary encoder signals for manual settings. The reported position changes by every encoder step. The faster you turn the encoder, the larger the position change at every step.

RotaryEncoder.zip

[terjeio]

@jschoch , @HuubBuis Thanks for the code.

My next step will be to add MPG support in the encoder plugin in such a way that it should be easy to add/test algorithms. If this is possible to do as I would like it then algorithms should be possible to implement as a single function. Hardware stuff taken care of at the driver level, and housekeeping in the encoder plugin framework.

When this is ready it would be nice if I can get help with testing and coding of algorithms.

[phil-barrett]

This sounds good. I will be playing with it for sure.

[HuubBuis]

I will add an mpg connector to my new controller board despite I am scary for a MPG that is directly connected to the controller.

[terjeio]

I will add an mpg connector to my new controller board

Strictly speaking you are adding an encoder input, this can be used with the encoder plugin for two purposes, either overrides or as a MPG. It can also be used for input for spindle sync but this will not be handled by the encoder plugin.

When used for overrides the plugin adds real time commands to the override queues exactly as input from the serial stream does.

I am scary for a MPG that is directly connected to the controller.

When used as an MPG the plugin will send gcode to the main protocol loop via a HAL entry point that can only enqueue a single block (or line), it does not interfere with the serial stream at all. The entry point will currently only accept gcode when grbl is in IDLE, JOG or TOOL CHANGE states. I have been using this entry point from the I2C keypad plugin for jogging for some time now and I am pretty confident it is behaving. If you are afraid that MPG input could still interfere with a running gcode file then it is also possible to block input completely by starting the file with a % character.

I am also thinking about disabling interrupts from the encoder interface when in MPG mode and code is running, this to reduce overhead.

[HuubBuis]

Strictly speaking you are adding an encoder input, this can be used with the encoder plugin for two purposes, either overrides or as a MPG. It can also be used for input for spindle sync but this will not be handled by the encoder plugin.

For me, running a cnc lathe without threading is not an option. A MPG will use quit a lot of IO pins. This could be a problem for the controllers I have set my mind to.
The mpg encoder will be turned manually and even a 1024PPR encoder can be handled by interrupts on the input pins. When the mpg is used, the controller will not be doing anything else. Spindle sync and MPG should be possible.

If you are afraid that MPG input could still interfere with a running gcode file then it is also possible to block input completely by starting the file with a % character.

Then when the gui sends a % character, the mpg will be blocked to!

A am pretty sure that in the end, I will connect the mpg to the gui, that suits me better. But who knows, time will tell.

[terjeio]

A MPG will use quit a lot of IO pins. This could be a problem for the controllers I have set my mind to.

Ok, good point. You are surely aware of some MCUs having quadrature encoder (decoder?) peripherals, these can reduce interrupt overhead substantially if position is read by polling. TM4C123, iMXRT1062 (Teensy 4.x) and SAM3X8E (Arduino Due) are among them.

Then when the gui sends a % character, the mpg will be blocked to!

Yes, until the running gcode is completed. E.g. a M30 will unblock the MPG.

A am pretty sure that in the end, I will connect the mpg to the gui, that suits me better.

IMO it is good to have a number of options, e.g. my MPG & DRO provides input to the controller via a secondary input stream and can be used to run the lathe for simple tasks without firing up the GUI. The encoder plugin will be another.

[terjeio]

When encoder is set to Universal mode (for overrides) the selected mode will now be reported in the next real time report and used to set the background color of the corresponding field in the next release of tsender:

A message will also be sent on a mode change.

I have now implemented very rudimentary jogging support in encoder.c, two tentative modes as of now. I do not currently have any hardware to test this on, at some point I need to make a test jig for that. Since I have made all the housekeeping code common it should be fairly easy to experiment with different jogging algorithms - each is implemented in its own function.

@jschoch : as you wrote your code was hard to follow, initially I am interested in how often polling of the encoder is done to calculate velocity.

@HuubBuis : I have not looked in detail at your code yet, the encoder interface code will typically be in driver.c and may get its input from a MCU peripheral so of limited use there. I will soon look into how encoder velocity may be used to modify the position, then likely in encoder.c - I will check out how you have done that then.

Currently only the T4.x driver has code in driver.c to support encoder input, for now limited to one and it does not use the ENC peripheral. If anybody is willing to experiment with encoder jogging but for a different MCU add a comment here.

I will commit this to the test branch a little later as I have to verify all drivers for step pulse length change from int to float first.

@phil-barrett : The code commited to my Subversion repository is up to date for T4.x for you to play with.

If anybody else wants read access to the "bleeding edge" in my Subversion repository for test purposes add a comment here. Note that access will only be granted to those who has shown an interest in participating and need it for adding/testing new features.

[HuubBuis]

Currently only the T4.x driver has code in driver.c to support encoder input, for now limited to one and it does not use the ENC peripheral. If anybody is willing to experiment with encoder jogging but for a different MCU add a comment here.

I will test the encoder jogging but for that, I have to finish the new controller design of my lathe. As said before, that will be around September.
Testing jogging can be done during the normal turning jobs so basically that is a no brainer. A showstopper will probably be the lack of threading (G33). I haven't looked at how you implemented this, but it should be easy to implement this on other hardware. If not, maybe that is something we could improve.

I haven't decided on the MCU and IDE yet, but it has to be something that can easily be flashed even by novice users. It took me 20 days to get the ESP32 version compiled (Grbl_ESP32 & Arduino 10 minutes, SAMD21 & Atmel Studio 10 minutes, STM32 & STM32Cube 15 minutes). Currently I am evaluating Segger Embedded and are waiting for some SAM D21 mini boards.

I have definitely chosen the Segger J-link as the new debugging probe and ARM as the new processor family.

[terjeio]

@HuubBuis As you know I have implemented spindle sync for the MSP432 driver, and done so at the driver level using a PID loop. I am not using a quadrature encoder for this as myself I have no need for moving the Z-axis when manually turning the spindle. Spindle sync with manual turning the spindle will add complexity to the grbl core that needs (a lot of?) work...

For the MSP432 I am using timers in a way that does not generate interrups for each pulse from the encoder, the encoder pulses feeds the clock input to a timer instead. The PID loop is fed by the actual and expected distance at the start of every segment (approx. every 5ms) and the feedback signal used to adjust the segment time, this without the grbl core knowing about it happening.

The encoder plugin will not be used for spindle encoders, I have tinkered with moving the MSP432 code out into a new one but not yet managed to do so in an elegant way. I will revisit this when I add spindle sync to the next driver, likely for the Teensy 4.1 or the BlackPill (STM32F4xx).

I haven't decided on the MCU and IDE yet ...

I see that you are pondering the SAMD21, I would not choose this for the lathe as is does not have a FPU and has some weirdness about its peripherals I have not seen on any other processor. When writing to many registers one has to wait for a sync bit before continuing execution...

Just before writing this I was trying to update the pulse width handling for the new float setting in the SAMD21 driver, but I was unable to get the pulse width below ~4 µS which IMO is weird, interrupt latency should not be this bad? I'll try again tomorrow...

I agree that a debugging probe is a must for any serious developer - I have wasted countless hours on cards that does not have a SWD or JTAG interface. Perhaps I am bad at writing code...

[HuubBuis]

I have no need for moving the Z-axis when manually turning the spindle

I don't need that to. Just an index pulse to always start at the same spindle position so that it is safe to stop and turn the spindle in any direction (measuring).

I see that you are pondering the SAMD21, I would not choose this for the lathe as is does not have a FPU and has some weirdness about its peripherals I have not seen on any other processor. When writing to many registers one has to wait for a sync bit before continuing execution...

For me, to develop, there are 2 major requirements:

• A decent IDE and source control
• A board that has a debug connector
  For the users of grblHALL 1 requirement
• A board that is easy to flash, even for novice users

I have chosen the SAMD21 mini (for evaluation) because it has a decent debug header and a small footprint. The next board to evaluate could be Arduino Nano 33 IOT because it has WiFi onboard or Duo or etc.

For me this SAMD21 (or any other Atmel arm processor) is just an arm processor supported by the IDE I like the most, Atmel Studio. For VScode you have to learn a lot of keyboard shortcuts (horrible) and eclipse has no icons for so many functions i most frequently use.
A big problem is GIT. I can't get GIT working the way I like. My basic libraries are used by almost every program I have made. Some Atmel code files are also used in VisualStudio. Probably due to my lack of knowledge, but I just can't get GIT working like Visual Source Safe. I could still use VSS but then it is not integrated in the IDE and that is not productive.

Segger Embedded is the best IDE alternative I have found so far but Atmel Studio is so more productive for me. My time is limited and I don't want to be slow downed by the IDE.

Within a few weeks, I have to decide what hardware and IDE it is going to be. Until then, I can play some more.

I agree that a debugging probe is a must for any serious developer - I have wasted countless hours on cards that does not have a SWD or JTAG interface. Perhaps I am bad at writing code...

Without a debugger you spend a lot of time writing code to debug. It is better to spend this time on the program to write.

By the way, I have decided to use the ESP32 for my rotary table (ATC).

[terjeio]

@HuubBuis

... in the SAMD21 driver, but I was unable to get the pulse width below ~4 µS ...

Big mistake by me when creating this driver, I have used Arduino code for handling GPIO pins resulting in a huge latency for setting step outputs (500 ns per pin), and it will be a while before I am going to correct this. Or maybe you want to correct that as a challenge?
What I did wrong was to use Arduino pin numbers for the mappings, translating those back to register addresses and bit masks is what causes the overhead.

I opened a new issue for spindle sync discussion a while ago where further exchanges about that topic should continue. As for disussions about processors, IDEs etc. a new issue/issues should be opened in order not to drown out the original topic of this issue.

[HuubBuis]

Big mistake by me when creating this driver, I have used Arduino code for handling GPIO pins resulting in a huge latency for setting step outputs (500 ns per pin), and it will be a while before I am going to correct this. Or maybe you want to correct that as a challenge?
When the SAMd21 arrives and I can test it on the bench, I will give it a try.

I have thought very long about your spindle sync implementation. I am still not sure what to expect. Maybe, after finishing the latency problem, it is time to implement G33 as well. Then, when I make an interconnection board for it, I can test this board for a long time on a lathe.

I opened a new issue for spindle sync discussion a while ago where further exchanges about that topic should continue.

I know you did. My main and only concern is that is could be difficult to implement on others hardware. As soon as I have the time, I will look in detail how it is implemented in the teensy board. That will give me some time to think about it before I start implementing and/or testing it.

will be continued...

[terjeio]

will be continued...

Over at the spindle sync discussion.

[terjeio]

The test branch has been updated with the latest changes. See the ReadMe for details.

I will be busy for the next 5-7 days, so not much time for coding.

[hdo]

I would really like to see MPG encoder support for the STM32F4xx device. Also willing to do the testing ;-)

[terjeio]

@hdo Any suggestion for which pins to use? It cannot be on pin numbers (regardless of which port) already assigned to an interrupt handler. Another option could be using the inbuild encoder interface which I saw mentioned in the data sheet (see 12.2 TIM1 main features) - have to dig deeper to see which pins are available for that.

Also willing to do the testing ;-)

Coding reasonable algorithms for MPG will be the hard bit, can you help with that too?

[hdo]

@terjeio

Any suggestion for which pins to use? It cannot be on pin numbers (regardless of which port) already assigned to an interrupt handler. Another option could be using the inbuild encoder interface which I saw mentioned in the data sheet (see 12.2 TIM1 main features) - have to dig deeper to see which pins are available for that.

Currently i'm using the STM32F103 version but i'm eager to switch to the STM32F411. So pin mapping is not finally set on my side.

However i think it would also be fine to implement something like your booster pack version for the TI microcontroller ;-)

[terjeio]

However i think it would also be fine to implement something like your booster pack version for the TI microcontroller

I am considering updating my existing boosterpack "motherboard" design and making a small batch for sale. The current design is for the F103 and lacks encoder input headers. It has a spindle PWM to DC converter and SD card socket on board already and can be used with both the "plain" boosterpack as well as the one for Trinamic drivers. I need at least one myself if I go ahead and implement spindle sync for the F4xx driver.

[einencool]

@terjeio
Hello Terje, I want to make a new Firmware for the Teensy 4.1 Board and want to set the encoder to Point 1 (Feed Rate adjustments)

In which File I have to do the changes? In the wiki I can't find it

[terjeio]

In which File I have to do the changes?

I have missed committing the empty encoder plugin folder to the driver. It is similar to the eeprom plugin, copy the encoder folder from plugins to the main/src folder and enable it in driver.h by setting QEI_ENABLE to 1.

I have not assigned pins for the default driver yet, these has to be added as well. This is how I did it for my BoosterPack board:

#if QEI_ENABLE
    #define QEI_A_PIN      GPIO0_PIN
    #define QEI_B_PIN      GPIO3_PIN
//    #define QEI_INDEX_PIN  GPIO2_PIN
    #define QEI_SELECT_PIN GPIO1_PIN
#endif

Perhaps @phil-barrett can tell which pins are best suited quicker that I can...

I have just commited the missing encoder folder to the test branch.

[einencool]

Oh, that was not the problem, the encoder was working already on the last release, but you mentioned that i can set the function to adjust the „feed rate“ and with a single or double click on the encoder i can set it back to 100%, that’s what i meant with option number 1 :-)

[terjeio]

Ok, too many things spinning in my head today... Sorry for that.

Settings are $400 for encoder mode, 1 is Feed rate override.
$401 is count per revolution.
$402 is count per detent. I may change this to a checkbox later. Valid values are 1 and 4.
$403 is double click sensitivity in ms. This is only relevant for mode 0 - universal.

I'll add this to the wiki later when the options needed settle down.

[einencool]

Ah perfect, thank you.
I hope I’ll get some time to try tomorrow, after repair of my cnc and wiring the Alarm circuits...

Wish you a nice weekend, here in germany we have around 34 degrees, and that’s way too much for me...

[einencool]

@terjeio
I wanted to give you a little Feedback after I got the new Firmware onto the Teensy.
For $400 I used the Mode „1“ and it works like it should, the only thing is, that the direction of the encoder was switched from the last „Master Branch“.
But I‘ll change it in the Config, then it should work like it should 👍

[phil-barrett]

I believe this has been resolved to everyone's satisfaction.