Better automate manual water rewards

[hanhou]

We'd like to improve the algorithm of autowater delivery for motivation and bias

1. Motivation

   • Current: give autowater when N ignore trials happen in a row, where N is defined in the curriculum:
     
     
   • New proposal:
     • TBD

2. Bias

   • Current: implicitly addressed by giving autowater when N Unrewarded trials happen in a row
     
     
   • New proposal:
     • Inspired by Bowen's comment (and Han's experience)

       Start with manual water to overcome bias. Continue pressing manual water for a couple of trials on the choice side.
       If animals switch, then monitor closely their choices and block transitions in next a few blocks.
       If animals don't switch and continue sticking to one side, move lickspout at a step size of 100µm. And only move lickspount for 1 step size per trial away from the biased side.
       Move the lickspouts until animals switch to the other side while pressing manual water on the previously biased side in a couple of trials to ensure animals can still reach the biased side.

      (autowater only)
      while auto_water_for_bias == True:
          If |bias| >= `threshold_1` and P_L != P_R and last `N` choices are all on the side with lowest probability:
               give `M` drops of water on the better side
               wait for `X` trials
     

• related https://github.com/AllenNeuralDynamics/aind-behavior-blog/discussions/891

[ellahiltonvano]

@micahwoodard tagging you since this overlaps with automation of lickspout movement and manual water

[micahwoodard]

@alexpiet @hanhou @XX-Yin would this be a good place to discuss the algorithm for auto watering for bias? Is this algorithm what we want to continue with for automating this?

(autowater only)
while auto_water_for_bias == True:
    If |bias| >= `threshold_1` and P_L != P_R and last `N` choices are all on the side with lowest probability:
         give `M` drops of water on the better side
         wait for `X` trials

[ellahiltonvano]

@alexpiet @hanhou @XX-Yin will you be involved in automating manual rewards? If so, can you provide feedback here for Micah?

[alexpiet]

Yes @micahwoodard that algorithm makes sense.

I would also include some check that we aren't giving auto-water and moving the lickspout at the same time

[ellahiltonvano]

@hanhou @XX-Yin we are still waiting on feedback for Micah's proposal above

[micahwoodard]

@alexpiet what should be the logic if the auto-water and lickspout movement are happening at the same time?

[alexpiet]

I'm not sure. Something reasonable like if one is happening, delay the other?

[micahwoodard]

I'm not sure. Something reasonable like if one is happening, delay the other?

@alexpiet do we want to have like a buffer between the two? So say like we want to wait at least 20 trials between interventions? So if lickspout moved at trial 50, no manual water until trial 70?

[micahwoodard]

@hanhou @alexpiet @XX-Yin what would be good default values for N, M, and X? And should M be number of drops or volume?

(autowater only)
while auto_water_for_bias == True:
    If |bias| >= `threshold_1` and P_L != P_R and last `N` choices are all on the side with lowest probability:
         give `M` drops of water on the better side
         wait for `X` trials

[alexpiet]

M should be volume. I'm not sure about default values. @ellahiltonvano @ZhixiaoSu

[ellahiltonvano]

M should reflect the current reward volume for a given stage (usually 2uL).

N: if the animal chooses the lower probability side without sampling the higher probability side for ~10 trials, give 2uL water to unchosen (higher probability side)

Wait for 5 trials, and then apply the same logic to deliver water again.

I'd say that we would only want to do 1-3 drops, and then if the mouse isn't responding move the lickspout so the higher probability side is closer to the mouse.

Bowen summarized his process here, which reflects how Han previously trained mice as well https://github.com/AllenNeuralDynamics/aind-behavior-blog/discussions/891#discussioncomment-12049434

[micahwoodard]

I'd say that we would only want to do 1-3 drops, and then if the mouse isn't responding move the lickspout so the higher probability side is closer to the mouse.

How do you decide how many drops? Why not capture that in the volume? Or are you saying give water for 1 to 3 trials where the conditions are met and then stop?

And then you are proposing instead of a buffer between the two options, default to water first and if that fails, try lickspout and keep that cadence throughout the session?

if the mouse isn't responding

does this mean if the mouse doesn't lick the other lickspout?