Yes, there is a great great chance of decreasing your email sending reputation and also over time even your Valid emails will bounce over time and your emails will land more often on Spam.
Because of this, the best practice is to use a Validation Tool. This way you will skip a large headache.
Let me ask you a question; Is it easy to walk alone in the jungle without getting harmed?
Answer: There is a 99% chance of getting hurt and 1% for not getting hurt.
That is also true for Email Verification; Without validating your email list, it is as dangerous as going alone in the jungle because If you don't verify your email list from trusted email verification vendors, you will burn your domain reputation and IP reputation very fast.
And if you are sending emails to invalid email addresses, it is not possible to get high value or high returns from your email list.