How to Fill Small Cracks in Concrete - For Dummies. Filling small cracks in concrete keeps them from turning into big holes. If you know how to fill small cracks in concrete, you’ll restore your concrete to a smooth finish. This technique will work for cracks smaller than 3/8 inch wide in the sidewalk, driveway, garage floor, basement, or steps.