⬆️ Update website

Author: dumindu

docs/robots.txt

@@ -0,0 +1,42 @@
+User-agent: Googlebot
+User-agent: YandexBot
+User-agent: baiduspider
+User-agent: Applebot
+Allow: /
+
+User-agent: *
+Disallow: /docs/borrowing/
+Disallow: /docs/cargo-crates-and-basic-project-structure/
+Disallow: /categories/
+Disallow: /docs/combinators/
+Disallow: /docs/comments-and-documenting-the-code/
+Disallow: /docs/control-flows/
+Disallow: /docs/crates/
+Disallow: /docs/custom-error-types/
+Disallow: /docs/
+Disallow: /docs/enums/
+Disallow: /docs/error-and-none-propagation/
+Disallow: /docs/functions/
+Disallow: /docs/generics/
+Disallow: /docs/hello-world/
+Disallow: /docs/impls-and-traits/
+Disallow: /docs/installation/
+Disallow: /
+Disallow: /docs/lifetimes/
+Disallow: /docs/modules/
+Disallow: /docs/operators/
+Disallow: /docs/option-and-result/
+Disallow: /docs/overview/
+Disallow: /docs/ownership/
+Disallow: /docs/panicking/
+Disallow: /docs/primitive-data-types/
+Disallow: /docs/smart-compiler/
+Disallow: /docs/std-primitives-and-preludes/
+Disallow: /docs/structs/
+Disallow: /tags/
+Disallow: /docs/unwrap-and-expect/
+Disallow: /docs/use/
+Disallow: /docs/variable-bindings-constants-and-statics/
+Disallow: /docs/vectors/
+Disallow: /docs/why-rust/
+Disallow: /docs/workspaces/