To check your sitemap for errors, use Screamingfrog to configure it. Open the tool and select List mode. Insert the URL of your sitemap.xml to the tool by uploading it, then selecting the option for “Download sitemap”. Screamingfrog will then confirm the URLs that are found within the sitemap file. Start crawling and once done, export the data to CSV or sort it by Status Code. This will highlight errors or other potential problems that you should head on out and fix immediately.
However, this does not mean you cannot topple them. It just takes more of an effort in terms of content as your page has to build the trust. That is why you will see the "Google dance" happening for fresh content from a site that is not yet trusted or is not very authoritative. Google gives your page a chance and measures user click-throughs when it pushes you to certain spots in the SERPs and then measures user engagement levels when the traffic hit your site through those positions in the SERPs.