Printer Friendly Version
Email this thread to a friend
|
Featured Web Site Template |
|
Reflects user activity within the last 5 minutes
|
|
| Member |
Message |
Prowler
Staff
Joined: Aug 14, 2000
# Posts: 1788
|
Posted: 2008-Aug-02 09:44
Good that you found the cause freeflyer. I hope that is the only reason.
When it comes to running Xenu, many use a high parallel thread than the situation warrants. If you run 50 threads in parallel and assuming that you have a good connectivity (T1 or better) you will pose lots of spikes on the server load when it comes to dynamic pages. Any automated tool generally puts on a high transient load on the server. Unless the scripts are optimized or use caching, this can bring throttling of server resources for most shared hosting environment.
The skewed results may not be the actual normal duty results. A look at the Google webmaster panel for the crawl data would be essential.
|
 |
freeflyer
Joined: Aug 06, 2007
# Posts: 211
|
Posted: 2008-Aug-02 11:18
yes agreed. This is on a dedicated server though of all things.. admittedly its running about 40 sites, but none of them has major activity, and i even tried turning the high load sites off as a test, it made no difference. You'd expect a little more as its not even being run near its limit, or rather, it shoudnt be. Static pages are fine, but anything which uses mysql is sloooooooooow to return.
Does anyone else use donhost? They used to be superb, but now its problem after problem and slow networks. The pipex kiss of death.
It all begine to fall into place now though anyway, as I've been keeping an on google webmaster tools, and a lot of sites on that server frequently have to have the verification redone , presumably also due to timeouts ! If it cant even verify the site and find the verification .html page, what chance has it with getting to the pages and listing them effectively.
Anyway, although i've checked for timeout errors on google webmaster tools and it comes back fine, i'm not convinced, as a lot of the pages are blank in the cache.
I think whats happening is that the page request itself is being returned fairly quickly, but the time taken to parse it is far too slow for any effective indexing. Google is simply leaving rather than waiting eight seconds or more for the page to fully appear.
I'll know in about ten days i reckon as to whether or not this is all correct, but i'm fairly hopeful at this stage.
|
|
g1smd
Staff
Joined: Jul 28, 2002
# Posts: 10418
|
Posted: 2008-Aug-02 19:23
What do the graphs in the Crawl Stats section of Google Webmaster Tools say?
One of the graphs shows you the average page load time. I get 0.2 to 0.6 seconds for all my pages, but then hit the odd day when it spikes to 1.6 to 2.2 seconds.
I have previously seen other sites running at some 4 to 8 seconds, and regularly like that for most pages on most days. I would consider that to be a problem.
|
|
freeflyer
Joined: Aug 06, 2007
# Posts: 211
|
Posted: 2008-Aug-04 10:57
crawl stats say maximum 8seconds, average 3 secs. Not good. I cant imagine a bot waiting much more than 3 secs for a page to load.
|
|
Prowler
Staff
Joined: Aug 14, 2000
# Posts: 1788
|
Posted: 2008-Aug-04 11:30
Whilst on the topic of server response time, have you tried this Firefox addon Firebug + Yslow ? This tells you the time to load various components in your page.
|
|
g1smd
Staff
Joined: Jul 28, 2002
# Posts: 10418
|
Posted: 2008-Aug-04 12:23
That data is also available under the Develop > Show Network Timeline Options in Safari.
Both Firebug and YSlow for Mozilla Firefox are also great.
|
|
freeflyer
Joined: Aug 06, 2007
# Posts: 211
|
Posted: 2008-Aug-04 13:08
yep tried them, although they dont tell me more than i can see, as i actually can watch eight seconds ticking away before a page is fully loaded !
Looking into this more deeply though, this is kinda interesting.....
I've got another site on this server which i optimise for. running xenu shows the internal links to one of the main top level category pages (of which there are only about ten of) to be around 1000, as there is a link to it on every page. Google webmaster tools shows around 0 internal links to this page. These pages have been up for six months or more, which is plenty of time for google to find the links to them. I'm finding it odd that it isnt...
|
|
acwebguru
Joined: Jul 14, 2007
# Posts: 24
|
Posted: 2008-Aug-25 11:10
I think that it should need to re-optimize your site again,and keywords that you are using now need to be research according to there search volume and promote them into quality and related web sites.
|
|
freeflyer
Joined: Aug 06, 2007
# Posts: 211
|
Posted: 2008-Aug-28 19:29
i think you're talking twoddle 
read the thread again.. this is way past keyword research.
|
|
You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
- You have not yet logged in, or registered properly as a member
- You are a member, but no longer have posting rights.
- This is a private forum, for which you do not have permissions.
If you are a recent member, it's possible that you simply have not yet confirmed your account. Please
check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions
contained within.
If you cannot find this message, click here to Re-Send it.
|
If you are still experiencing problem, please read the
Login Assistance
Article for some advice on what may be causing your login not to work properly.
|
Switch to Advanced Editor and ...
Create a New Topic
or Reply to this Thread
|
|
Inbox
