How To Get A New Site Listed In The Search Engines
- 30 Comments... Click to Contribute
This is a problem I’ve never had to deal with yet watching a few friends recently start new websites has made me realize that getting a brand new site to be discovered (indexed, listed) by the search engines can be a difficult task. In my case since I’ve been operating online for so long whenever I launch a new site I simply link to if from my existing sites knowing in confidence that Google, Yahoo! and MSN Search will quickly index the site. But what do you do to get a site indexed if you have no existing portfolio of sites already established to take advantage of?
How Do Search Engines Work?
First you must understand how exactly the search engines find websites and index content. In order for your website to show up in a search result your pages must exist in the search engine’s index. Each search engine has an absolutely massive index (let’s not talk about comparing index sizes, it’s like a search engine mid-life crisis joke waiting to happen) and it’s constantly updated every minute of every day. The search engines have what are called bots or crawlers or spiders, basically computers that ‘pretend’ to be browsers surfing the web devouring content like hungry monsters. There job is to make sure the search engines have the most up to date index of webpages possible so that when a person uses a search engine they get the most relevant response available. Obviously it’s a lot more complicated than that but I’m into simple explanations and that’s really all you need to know.
Your goal is to get the little spider to come visit your site and index every page of content so as to maximize your search engine results page performance. In order for a spider to crawl the web and visit your site there must be an incoming link pointing to it. So what’s the best way to accomplish this? Read on…
Submit Your Site To Directories
One of the paths of least resistance to get an incoming link is to use a directory. Directories exist to simply list websites, usually broken down by category. All you need to do is fill out a form and depending on whether submissions are manually or automatically approved, your link will show up in the directory with a period of time, usually 24 hours.
The problem with directories is they are essentially link farms, no content and lots of outbound links. Consequently they usually have very little clout in the search engines and since your link is sitting in amongst possibly thousands of other links the little search engine spiders may never find your site.
In order to maximise your chance of success using this technique choose some of the higher profile directories. If you read this post - Free Traffic From Free Directories - you will find a link to a solid listing of reputable directories that you can submit your site to.
Comment In Blogs, Forums, Bulletin Boards And Newsgroups
This strategy is a delicate one. You don’t ever want to SPAM - purposely post just to mention your website without contributing anything of value. Even if you think you are contributing something of value by linking to your site chances are your post will be deleted since people generally have a thing against self promotion, no matter how valuable your self promotion may be (or you think it may be).
The best way to go about this is to have a signature file (most forums have a function for this) that allows a few lines of self promotion to be included at the end of your posts. Enter the community, get involved in the discussion and contribute. Now obviously you will have more knowledge to draw from if you pick communities based on topics you are familiar with and preferably are relevant to your site. Not only do you improve your chance of search engine spiders finding your site, you might get some real humans clicking your signature and visiting your site as a bonus (this is in fact a good general online marketing strategy to get more traffic to your site - I use it regularly).
A note about blog commenting - thanks to the NoFollow tag which many blogs have on by default, your comments will not be followed by the Google spider. Read this article - Trackbacking Lies - How rel=â€Ânofollow†Stops Spreading the Love - for information about - for information about the NoFollow tag.
A better strategy with blogs, albeit a more labor intensive one, is to set up a blog yourself and start writing up good content. Head to some similar blogs talking about similar topics to your market, join in the conversation by posting relevant comments and you may find some other bloggers will link back to your blog. Most blog commenting systems allow commenters to make their name clickable to a website (my blog does this) so people will read your comment and then click your name to visit your website. When they come visiting and they find some good interesting content they may link to you in a blog post. If a popular enough blog links to you that will be all it takes to get the search engine spiders to find your site. Mission complete.
Press Release Distribution
This technique is easy enough to implement but faces the same issues as directories because so many people make use of it you may get lost in the crowd. You can go to a site like PRWeb, submit your press release about the news of your new website launch and they will publish it for journalists to make use of. A good press release may even circulate all over the Internet but at worst you will be at least published on PRWeb (there are others PR sites out there - try a search for “press release distribution” or similar).
I won’t go into too much detail about how to write a good press release, that’s whole article in itself, just remember to cover the who, what, when, where and why and try to come up with an interesting human interest angle. Note that this strategy is also a very good general marketing promotion technique so it might be worth doing simply for the experience and testing anyway.
Google Sitemaps
For the more technically savvy out there Google provides a sitemap submission system that gives you the tools to directly communicate with Google to let the search engine know when your site updates and ensure your entire site is indexed. This is only available for Google.
Manual Submission
Most search engines provide a facility to submit your site manually. Now while this may end getting your site indexed it really is just about a useless activity and could even slow down the process of becoming indexed. Remember that search engines rely on links to find, and more importantly, to evaluate sites (read my articles on search engine optimization if this is a new concept to you). Get yourself one relevant quality incoming link and that will do a lot more to get your site indexed quickly than manual submission ever could.
Some search engines offer a form of paid inclusion service that will speed up the process of getting your site indexed. Personally I think this is a waste of money. It might get you in the door but it won’t have you ranking highly for any search terms. It’s like opening a door to let you join the end of a very long queue. In order to advance up the queue you need to get incoming links. Incoming links will also open that door for you for free, so paid inclusion is really a waste of money and time. Focus on incoming links.
Link Loving And Paying For Links (Link Prostitution?)
Link love is a relatively easy thing to be on the receiving end of. You have probably heard of the favor bank, the social currency of our world. You can open an account with almost anyone by making a deposit by simply doing them a favor. Maybe you’re really good at graphic design so you can create a logo, or perhaps you’re an accountant so you can offer tax advice, or a lawyer could look over a contract or, well you get the picture. You can then make a withdrawal by asking for a link to your website. Favor currency flows quite freely and can easily be accrued by doing good deeds.
If you don’t feel like using the favor bank you can always try the other currency, cold hard cash, and buy some text link ads on other sites. A $50 link on a prominent site for a month should do the trick to get your site indexed. This is the rich man’s answer though and really should not be applied if your goal is only to get indexed, there are so many free ways to do it it’s hard to justify spending the cash. However if it forms part of an overall online marketing strategy that you intend to put into place anyway you may as well start spending straight away and kill two birds with one stone.
It’s All Part Of The Online Marketing Process
I’ve only barely scratched the surface of the many ways you can get your site indexed. Really it’s one of the easiest tasks, so don’t stress about it for too long. If you have the intention of putting into place smart online marketing practices and learn as you implement, then the initial indexing step will not concern you. The overall search engine optimization process takes a lot longer and you will quickly realise that just getting into the index is the very start of a long hard push, trying to raise your website from the very bottom of the index to the top result pages for your key terms.
Yaro Starak
Search Engine Optimizer
Subscribe to Entrepreneurs-Journey.com
Free with subscription
"How To Start An Internet Business
& Make Your First $1000 Onine"
Forward to Friend
Email a copy of this article to a friend
Loading ...
Hi Yaro,
another great article and fantastic timing for me since I have just put my blog online in the last week, so I now have exactly the problems you mention here!
I have noticed that my site is already in Google, but only if I use the entire URL (minus the http://www. at the beginning and the .net at the end) as a keyword.
One interesting point is that my name, which is unusual, brings up your site top in google; namely for a previous comment I made.
So my name has become a unique search term for entrepreneurs-journey!
Forgive me if I try and pull it back towards my site, although that might take some time
Thanks for the article,
Hugo
Great post. Just a quick note on Google sitemaps. The sitemap protocal has been released under the Creative Commons License so other search engines can make use of it if they wish. Lets hope they pick up on it.
Hey, great post. And following your advice here is a post on my website that is totally relative to yours. It describes a trick you can use to get every page on your site not only spidered but also indexed in Google in only a few days.
Love your blog!
Miles
There are so many ways of getting indexed its not even funny.
If you are launching a comercial site and need it indexed instantly the best bet is to just buy a link from the Yahoo directory for $300.
BUT, most people have no need for that.
One great way to build you incoming links is by submitting articles to articles directories for other webmasters to use as content.
This will build up your visability online, and incoming links.
Not to mention that those sites get so much stuff moving through them they are indexed daily.
As long as you get Google, thats the important one.
MSN is index crazy and they will scout out your site even if its not mentioned anywhere….they are very good at that.
I learned the hard way when a thank-you page for one of my products was indexed by them even though it had no incoming links…
But its important to note that you should have the incoming links be very related otherwise your PR could suffer, not that it will though.
If you get your site linked to from slashdot or boingboing it will get indexed ASAP.
Also you can use the Digger service and get your links indexed right away with no funny business.
But like you said, scouting out related links from people with similiar sites is the best method and exchanging links with them.
Blogs are link hotbeds, with the no_fallow tag off of course
Hi Yaro,
This is a topic that I see often in the marketing forums and Michael Mayhew has one of the simplest methods I’ve found. It worked for me!
Check it out (for Google)
http://webmasterautomation.com/blog/?p=23
He has another for Yahoo but I haven’t had as much success with it…
Thanks for your hard work,
Mark
Great read…BUT….even doing all this and MORE, you can still be waiting, like me, for many months. I launched a site launched on 27 July 2005 and am still waiting to be even indexed by Google, let alone appear in any search results. The bot visits every day, but still no results. Even Miles’ theory is contradicted by my experience.
MSN etc are no problem. Only Google.
I’ve just tried all the methods (including the ones in comments), so fingers crossed it’ll work.
There is definitely a problem with re-using an expired domain. I have heard it can take about 6 months to get indexed. Please let me know if anyone has any tips for expired domains.
Hello Yaro,
Thanks again for such an informative blog, I really appreciate it.
I have a question about sitemaps. I just started using the Google Sitemaps plug-in for my Wordpress site. It creates an xml file which I placed a link to on my sidebar. I noticed for your site that the sitemap is an actual web file. Am I ok with just an XML file for a sitemap or do I need to set up an HTML sitemap similar to your format?
Thanks,
Knox
Knox - As long as you actually submit the XML to Google sitemaps it should be fine https://www.google.com/webmasters/sitemaps/login
My site is well spidered by the googlebot because it has over 10,000 incoming links so my HTML sitemap generally does a good enough job. I probably could use the XML hand submission method if I wanted to be absolutely 100% about it though.
Yaro,
What a great post! I’ve just started my own blog and am a complete novice at all of this. I will be sure to try out your methods, I’m sure they’ll help greatly. Thanks again.
Brian
I stepped into your website by mistake, now I can’t stop reading all the articles, so many interesting imformations, thanks for sharing.
TheMystical
If you are planning to get listed sooner in google do not submit your site to them. Make sure to be linked to a regularly crawled website. That is it.
Wow…very informative. I’ve always been perplexed with that Catch 22 when I’ve tried to get my blog off the ground. Sort of like applying for a job when they want experience, but you need the job to get experience.
Brendan
I came upon this site by accident and i must say, very interesting… most blogs are boring with boring content but this some real informative information. I think this is great stuff… keep it up… You got yourself a new reader…
cheers.
Opio - glad you liked it and thanks for your comments. You will find plenty more articles like these in the articles section (click the tab at the top).
Yaro
As a graphic designer I basically agree with most of what you had to say. Like many others I’ve been having trouble getting Google to index my site. All the other search engines recognize my site and it’s gradually crawling up the Alexa charts. Even with blog, original content, sitemap, and SEO optimization Google simply refuses to recognize my site.
But that’s ok, fortunately, I don’t need Google. I’m getting enough traffic and business from the other sites to more than compensate for Google’s PMS behavior. I’m also recommending to all of my clients to forget about Google and to instead use Clickbank to generate income for their sites.
No sense in getting paid pennies by Google when you can earn $25 or more through Clickbank. Or 100% commission with your own products. My point is that you can exist very profitably on the Internet without Google and the more people who get fed up with Google’s antics the more likely those of us who work so hard to create substantial websites will be recognized by the other search engines.
GrafxExtreme.com
Quick comment about GrafxExtreme. Microsoft’s new LIVE.COM has my site listed and indexed already. So much for Google. Looks like my graphic design business is going to continue.
GrafxExtreme.com
Hello Yaro and fellow bloggers,
Thanks for sharing your expertise about getting a new site or blog listed with the search engines with the rest of us amateur bloggers.
My issue with search engines has been and still is getting all my site pages indexed or at least the majority of them! Google sitemap and links between my sites (indexed + PR4) was no breakthrough.
I was amazed after having started my Hawaii blog with blogger in July this year to see my posts being indexed within 1-2 weeks!!! Curious how long it will take to get a PR for the blog. Any experience with blogger and PR?
Aloha, Pua
Hi Again Pua,
When it comes to deep indexing - ensuring every single page of your site is indexed - it’s important to also build links.
While doing things like having sitemaps and internal links helps, the search engines also take into account your quality score, which is a way of assessing how authoritative your site is based on the authority of links coming in, the frequency of the links, the age of the links and the frequency of new content.
If you don’t actually build links your quality score will be low and as a result the search engine spiders won’t spend as much time indexing your site and won’t visit as often.
To raise your quality score do two things
- Build links
- Add new content frequently
This will show the search engines that your site is an authority or at least working towards becoming one, and consequently you will enjoy more frequent spider visits and deeper indexing.
Yaro
Hello again Yaro,
Thanks for taking the time to specifically answer my recent question regards getting most of my 90 pages of my regular site indexed by Google.
I am always amazed when I see how many pages you and other people have come up with in their history of building their sites. I guess 90 pages is just the beginning.
I am definitely working on incoming links, however as I became more selective in choosing the right ones, it’s a slow process. We get link exchange requests on a daily basis but I turn most of them down unless they are from a quality site with good PR of the page for my link or Hawaii or travel related.
I also found out that a page can have a 3/10 PR but does not necessarily show a Google cached page. I think I mentioned that in your forum before. It’s mind-boggling to me. What I understood from other webmasters and SEOs: incoming links from pages not indexed or cached by Google won’t improve my site ranking in Google. They are like non-existing to the search engine! Right or wrong?
Love your forum and your newsletter. Mahalo and aloha, Pua
Hi Pua - in a nutshell - right!
However those pages currently not indexed may one day become indexed and make a positive impact on your site’s search rankings.
On the flip side, if those pages are banned and that’s why they are not indexed, you want to avoid being linked from them in case your site becomes associated with the banned pages.
My advice - keep blogging as a means to build links. It’s a much better way to build linkage than doing link swaps with low quality sites.
Cheers,
Yaro
Thanks for the great info as always Yaro. You have one of the most informative sites on many subjects out there. Anyone visiting can see that you have been in this for a long time, but your design of your site and your links within them. As far as getting your site indexed it took mine nearly 8 months, I went into this knowing it would not be an overnite success. Most people do not understand it does take time and content on their sites are so very important in PR rankings. Getting a sitemap was a great boost on my site, as were the experimentation with keywords and title tags. I am one who is willing to wait and continue to learn the SEO tactics to be a success. Best advise as always to people starting out is to not expect success overnite and just keep using the knowledge to create the power for Your success! Cheers and God Bless.
Thanks for dropping by Suzanna, and cheers for the compliments too!
It’s good to hear you are prepared to wait. Patience is a virtue when it comes to online business.
The easiest way to get indexed by Google is to make a free blogger account, and link back to your own site — then, you’ll be added nearly instantly.
Another quick, simple and free way to get indexed by Google and all the other search engines is to make a meaningful and on-topic post in a user forum, such as Digital Point. You’ll be picked by the search engines in no time.
Everyone is absolutely correct about Yaro providing essential information…and plenty of it! Some of you newer Bloggers might even want to check into Yaro’s “Blog Mastermind” study group…it is a terrific value and will get you headed down the right path a whole lot quicker than going it on your own!
Best wishes for everyone’s success!
Linda Martin-Peoples
P.S. Come to think of it, I believe even “seasoned” bloggers could benefit from a few sessions of “Blog Mastermind”…if for nothing else but to gain perspective.
Yaro,
Your website is a goldmine. I got into blogging about a month ago and thought Adsense was the end all, be all. Wow was I wrong. There is so much more to consider. After a lackluster first month of earnings I am going to implement some of your strategies and test your theories. I cant thank you enough for even more motivation to put my “free time” at my real job to “work” in something I simply enjoy doing (blogging).
Good Morning Yaro!
Wow..so glad I found your site. One of the best information tools I came across. I’m new to the internet world and your site is exactly what I’ve been looking for.
Sincerely
Paulette
http://www.parsnipdesigns.com
Thank you very mush for the article-Listing your new site in search engine.I learns a lot from it.