PLL lock graphic example

[VK4ADC]

Frequency drift is more likely the reference oscillator for the transmitter (and/or companion receiver if that uses the same frequency source) shifting higher as the unit's temperature rises with the power dissipation during the transmit cycle. The reference oscillator then drifts lower during every receive cycle, higher again each transmit cycle, et adfinitum. Typically such up/down drift is poor mechanical layout / design - proximity of the reference oscillator to the chassis and heatsink area.

It is not a PLL lock time effect that you are describing - that manifests as a quick jump from off-frequency to on-frequency in a matter of a second or far less - if it is even visible.  The frequency subsequently does not shift.