This is an old Jewish custom dating back to the medieval times or earlier. Placing a stone on a grave honours the dead by letting people know that the gravesite has recently been visited.
Another reason for using stones as opposed to flowers is that flowers wither and wilt, whereas stones are everlasting and do not die, thus the memory of the person will remain. Similarly, it is thought that in life, one enjoys material possessions and beauty, both of which are left behind when we die. Stones, being everlasting, represent spiritual wealth which lasts forever. In addition, it is probable that the custom originated in needing to reinforce the burial mound, which in earlier times would not have been a slab, or made of granite or marble.