You are 100% correct. The actual muscle damage (which is what stimulates muscle hypertrophy) actually occurs when you lower the weight. Lowering the weight is what actually tears the muscle fibres.
If you watch the guys with the best physiques at your gym train, you will notice that they all slowly the weight in a controlled motion.
You never want to drop your weights. You can lift the weight explosively, but lower it slowly under control to get the most out of your workouts.