Changes in temperature and pressure are one of the most common causes of irregular base line drift with refractive indexI detectors.  

If the column temperature is more than 20 °C above ambient, consider using a secondary heat exchanger to reduce the temperature of the mobile phase that exits the column to as close to the flow cell temperature as possible and also minimize any temperature fluctuations.

If pressure is the suspected problem, it could be a problem with the pumps, e.g. pump seals need changing. Turn off the pumps and see if the fluctuations are still observed. 

Finally, another cause of could actually be the lamp itself, usually tungsten or LED with RI detectors. If the lamp is weak, this can cause baseline fluctuations as well.