Verified the issues on the server and added some bot blockers to avoid website traffic.
Added Robots.txt file exceptions which include having the following permissions
1. Disallow: /app/ 2. Disallow: /bin/ 3. Disallow: /dev/ 4. Disallow: /lib/ 5. Disallow: /phpserver/ 6. Disallow: /pub/
This is included in the basic theme of Luma here we have covered the Luma theme and designed the website.
So this addition can be added to the Luma basic theme search robots section
Which can be mapped Content>Configuration>First theme edit option.
Thank you.