Krithiga
You are doing the right thing. Do not water (mist the sticks a little if they start to get dehydrated) and supply warmth on the bottom (place the pot in a warm place which it sounds like you are doing in a south facing window).
The plumeria are still dormant at this time of year so it will take a little longer. Do not disturb the cuttings to check them. You will know that roots are forming when the leaves are starting to grow. Be patient!!