Newsvine
  • Welcome
  • Help
  • Report Bug
  • Conversation Tracker
  • Your Column
  • Replies
  • Friends
Type Comments Since You Last CheckedArticle Source Last Checked Stop Tracking All Clear Tracking All
Advertise | AdChoices
Log In | Register
Close the Login Panel
Existing users log in below. New users please register for a free account.

New Users:

Existing Users:

E-Mail:
Password:
Forgot Password?
Please enter the e-mail address or domain name you registered with:
E-Mail/Domain:
Back to Login
Log Out
  • Top News
  • Local News
  • World
  • U.S.
  • Sports
  • Politics
  • Tech
  • Entertainment
  • Science
  • Business
  • Health
  • Odd News
  • More
    • Arts
    • Education
    • Environment
    • Fashion
    • History
    • Home & Garden
    • Not News
    • Religion
    • Travel
Visit Danny McGee's column >>

DANNY MCGEE

Home Page
Articles Posted: 34  Links Seeded: 13
Member Since: 1/2006  Last Seen: 2/22/2012

What is Newsvine?

Updated continuously by citizens like you, Newsvine is an instant reflection of what the world is talking about at any given moment.

Get a Free Account
Help
Fun Stuff
  • Your Clippings
  • Leaderboard
  • E-Mail Alerts
  • Top of the Vine
  • Newsvine Live
  • Newsvine Archives
  • The Greenhouse
  • Recommended Articles
  • Wall of Vineness
Put a Seed Newsvine link on your own site

A Humble Request for a New Feature

Thu May 29, 2008 9:25 PM EDT
technology, web-2-0, tech, newsvine, feature, meta
By Danny McGee
Advertise | AdChoices

As our community grows, the technology which fuels it must adapt. It's a feature employed by countless news sites, social networking sites, and web communities large and small. It's been overlooked by Newsvine thus far because, until recently, our tight-knit community has been too small--though not to its detriment--to require it. However, the glaring need for it has been exposed by a recent article seeded to us by MSNBC.com. It's a simple feature, so I will keep this brief and discontinue the fluff-for-the-sole-purpose-of-raising-word-count that has been this entire first paragraph.

We need some sort of functionality for splitting very large comment threads into multiple pages. I had never even considered this completely rudimentary feature as a necessity or even a neat thing to have until I visited this article after coming home from work tonight. I tried (and persevered to the bloody end) to post a comment on it, but was frustrated to no end by the fact that my browser kept lagging for a good 15 seconds for every 3 seconds I attempted to type.

I opened the task manager and discovered that, inexplicably, Firefox kept jumping to 100% CPU usage every few seconds. I figured maybe it was one of the other processor-intensive tabs I had opened, so I closed the rest of them (which in itself took way too much time and effort due to the intense lag). Finally only that single Newsvine tab remained, and yet the problem continued, which served to confuse me even more. Then I finally finished submitting my comment, was taken back to the top of the page, and noticed the comment count. Holy everliving crap is that a lot of comments. No wonder Firefox was making such a concerted effort to mutilate my processor.

Please, Newsvine. Implement this feature. Or at least consider it. Or at the very least, put it in the queue of new features to add later on down the road, when articles like this start to become more frequent. No, I am not using the most 1337 h4x0r system in the world, but I'm not exactly running on DOS, either. I'm running a Celeron 2.5 GHz processor with 768 MB of RAM. I can run Morrowind and Counter-Strike with respectable framerates. One would assume that my system would thence be up to the task of keeping up with an entirely text-based news website, but alas, it met with that beast of an article, it fought, it struggled, and it lost. I'd be genuinely surprised if I were the only one.

  • Enjoy this article? Help vote it up the 'Vine.

Back To Top | Front Page

Published to:

  • Danny McGee's Column
  • Groups: MetaVine
  • Regions: none
  • Public Discussion (35)
Danny McGee

So by a show of hands (well, comments), who else had this problem with the linked article? Maybe there's some sort of quirk with my computer and I really am the only one, but I really, really doubt it.

  • 6 votes
#1 - Thu May 29, 2008 9:26 PM EDT
kymlee

Heh, I just clicked the link and immediately my browser slowed to a snail pace. I was skeptical at first because, well I never thought about how an article with a large number of comments could slow the browser. As soon as I closed the tab though, I was back at full speed.

I've always hated multiple pages for one comment thread but it may be necessary - hopefully only for those with larger comment counts.

Thanks for bringing this to my attention. Oh yeah, and good to see you again. :)

  • 6 votes
#1.1 - Thu May 29, 2008 10:31 PM EDT
Paul William Tenny

Looking at the page source, I'm guessing it's the Javascript. I see 12 scripts referenced externally (that's just horrifically bad by the way) and several inline script blocks. There's probably a low-res timer running in the background to update the comment preview (x15 for other features) every half-second or so, along with god knows what else trying to processes and fill-in data while the page is still loading.

It may all work somewhat, but really it's very poor design, not well thought out at all. Problems that extensive and deep seeded I wouldn't expect to get fixed soon, if ever, unless Newsvine or MSNBC plunks down some cash to hire a development team to start over again. A good place to start is to stop using AJAX just because you can, because it's killing site performance.

  • 6 votes
#1.2 - Thu May 29, 2008 10:33 PM EDT
Tamh

took about 12 seconds to load the page, and usually I don't wait that long before going away... it's sad how fast we we expect gratification these days!!!

  • 3 votes
#1.3 - Fri May 30, 2008 12:23 AM EDT
Mike D.

Paul: Once again, you show a lack of knowledge about web development that astounds me, considering the conviction with which you like to write about it. #1: No, it's *not* the javascript... it's the hundreds of comments (as the author of this article correctly points out). #2: You're also wrong about external javascript calls... most sites, including ours, who use third party ad providers (see: everybody) require multiple JS calls to run asynchronously in order for ads to get correctly trafficked. #3: There is no poor design, there is nothing that wasn't well thought out, and there is nothing "extensive and deep seeded" that is preventing us from paginating comments... it's just another project of things to do in the queue, and one that has only gotten important now that we routinely have threads of over 500 comments. #4: What does AJAX have to do with any of this? What "AJAX" runs when you load a story page? Have you even looked at the source? I'll save you the time... the answer is "none".

And now begins the call to arms for Paul to write a 10,000 word comment trying to explain what he really meant...

  • 5 votes
#1.4 - Fri May 30, 2008 1:41 PM EDT
Allan Neal

Mike D, what does any of that have to do with fixing the tracker? : )

  • 1 vote
#1.5 - Fri May 30, 2008 1:46 PM EDT
spiffie

What does fixing the tracker have to do with this article?

  • 1 vote
#1.6 - Fri May 30, 2008 2:00 PM EDT
Allan Neal

Come now, every thread has a comment or hundred that veer off to related things. Right now there are a couple threads, Calvin started one, that are focused on new tools for the Vine. Calvin's got high jacked by the tracker problems. Another is dedicated just to the tracker.

If we don't fix the broken stuff first, are we really advancing? To hear someone at Newsvine talk about the technical side of things concerning new options and not explain what is going on with existing problems is frustrating.

The comment tracker has been temperamental for as long as I've been here and I've haven't heard one explanation from Newsvine.

The squeaky wheel gets the grease, which is appropriate since the comment tracker is definitely squeaky.

So what does it have to do with the article? This article is suggesting Newsvine throw into the hopper work. I want to hear that the tracker is already in the hopper before this one. I like this new idea, but the occasional long tread I can deal with. The time wasted reloading and following false alerts seems far more important an issue.

And I was trying to deflect a 10.000 word response from Paul that Mike D seemed not to want. I was doing a good job, I think, until you blasted me. Ah, but maybe that's what you were doing also.....

  • 1 vote
#1.7 - Fri May 30, 2008 2:25 PM EDT
Adam Hobson

and one that has only gotten important now that we routinely have threads of over 500 comments.

Of all the problems to have, scalability due to high usage isn't one of the worst ;-)

Mike, I just have one request to make, if you guys do implement some sort of pagination to deal with long comment threads, please figure out some way to make it work with those nice little "NEW" anchors on new comments. Being able to jump easily from new comment to new comment in a long article is one of the best things about newsvine's comment system.

  • 2 votes
#1.8 - Fri May 30, 2008 2:40 PM EDT
spiffie

until you blasted me

An eleven word question is "blasting" you? You must be new. ;-)

For what it's worth, I received a response from Newsvine indicating that they were adjusting some things with the tracker, and a couple hours after I received that response, my tracker problems went away. It's been working for me now as expected for about a day. I updated Gwenny's article last night with that information.

I would strongly recommend that if you're still having issues, you need to submit a bug report with Newsvine with as many details as possible.

  • 4 votes
#1.9 - Fri May 30, 2008 4:05 PM EDT
Adam Hobson

I think a big assumption we users make is that when newsvine does have a problem that the newsvine team knows about it. I'm sure that most of the problem they don't know about until we report them.

For example for the longest time I had a problem with newsvine crashing when I opened it up in multiple tabs in Firefox in OS 10.5. Never had a problem with it in Windows on Firefox. I ended up talking to a few users about it one time and we all experienced the same frustrating problem. The real problem though was that at no point did any of us ever report the issue to newsvine. The next day I reported it and Mark responded back within the day.

Whatever the tracker problem is, it's not site-wide because mine has been working fine for months with a hiccup a think once maybe for a day.

  • 3 votes
#1.10 - Fri May 30, 2008 4:30 PM EDT
Allan Neal

Well, I'm still learning the nuances of net speech. I'm not sure what really constitutes a blast. I did find it kind of "frontal". I guess that's good. I've never been really blasted.

I've done the bug report thing and a year ago when the fever was high, I assumed the old timers would take care of it.

So now you have the record: an eleven word blast. Stand proud, my man. : )

  • 1 vote
#1.11 - Fri May 30, 2008 4:37 PM EDT
spiffie

Mine had been working fine for months until this past week when it suddenly decided not to display all new comments when I visited a thread via the tracker. A refresh would display the missing comments. (I could tell there were "missing" comments because I would either show up at a thread with no comments, no new comments, or a discrepancy between the tracker's reported number of new comments and the actual number of new comments in the thread.)

But like I said, as of yesterday evening, everything was back to normal for me, shortly after I received Mark's reply to my bug report.

You're absolutely right, bug reports matter. The Newsvine guys are very rarely working off of comment threads for feature requests or user problems with the site.

  • 1 vote
#1.12 - Fri May 30, 2008 4:38 PM EDT
Paul William Tenny

Paul: Once again, you show a lack of knowledge about web development that astounds me, considering the conviction with which you like to write about it.

I'm not going to dignify that with a real response Mike, so let's stick to the actual discussion at hand and leave the trolling for another time.

No, it's *not* the javascript... it's the hundreds of comments..

Loading the page in question with a freshly launched browser and an empty cache each time:

Javascript enabled
Kernel: 03.381
User: 36.656
Total: 39.937

Javascript disabled
Kernel: 1.750
User: 24.531
Total: 26.281

Difference: 13.656

Bare minimum the out-of-control JS on this site is piling over 13 seconds of processing time on top of the already ridiculous 26 seconds it takes just to process the HTML. An extra ~34% is not nothing, even if it isn't the sole cause.

YSlow grade: F (25)
F: 1. Make fewer HTTP requests
   This page has 49 external Javascript files.

F: 9. Reduce DNS lookups
   This page has more than 4 domains.
   (25 listed)

F: 10. The following JavaScript files do not appear to be obfuscated nor minified.
   common.js: 34.4 KiB
   ajax.js: 2.1 KiB
   vine.js: 93.3 KiB
   spell_checker.js: 27 KiB
   s_code_remote.js: 15.8 KiB
   (Many more)

Next..

You're also wrong about external javascript calls... most sites, including ours, who use third party ad providers (see: everybody) require multiple JS calls to run asynchronously in order for ads to get correctly trafficked.

Of course they do, but their JS doesn't add 34% to the processing time of the page. Yours does.
As far as synchronicity goes, Javascript is only asynchronous in a networking sense, not in a CPU usage sense unless you're only targeting this site for multi-core users and assuming the IE6/7/8 and Firefox JSVMs are multi-threaded. And frankly I'm not wrong, since I never stated anywhere that large sites don't use multiple external JS files.

I quite clearly said that it was bad to reference a large number of them, and it is, especially when you're loading 49.

There is no poor design, there is nothing that wasn't well thought out, and there is nothing "extensive and deep seeded" that is preventing us from paginating comments...

Depends on how you look at the fix, since I was obviously talking about the way Javascript builds interactivity into this site and not comment pagination, you're basically making a different argument here that has nothing to do with what I said. (Which by the way is the second time you've done that.)

When your JS adds ~34% to the processing time of a given page, sure, you won't notice it at all on a small page. But as has been pointed out this isn't a small site anymore and many pages aren't exactly small either. This has expressed a flaw in the way you've integrated Javascript with the site or with the way the JS was written since it could simply be bad coding, but that fact can't be made to disappear simply by cutting the pages back into smaller pieces.

In all likely hood you're still going to be sitting on a +34% processing time no matter how small you make the page.

What does AJAX have to do with any of this? What "AJAX" runs when you load a story page? Have you even looked at the source? I'll save you the time... the answer is "none".

No I haven't looked at it because I'm not interested in sorting through 100+ KiB of somebody else's code to point out the problems for them. If you want someone to do that, you're going to have to pay them. Nor do I need to look at the code to know that the code is at the very least adding significantly to the processing load. If you want to dispute that, you've got to show me some numbers instead of making completely unsubstantiated claims.

And now begins the call to arms for Paul to write a 10,000 word comment trying to explain what he really meant...

More like "trying to correct the misrepresentations of what he said" since you basically got everything I said wrong, but if you're just going to be a caustic about it Mike, don't bother looking for the novel edition of this discussion. We had a good conversation on your blog on unrelated subjects once before but you're showing no signs of being able to do so again on Newsvine today.

I don't know if you're just reflexively defending the site out of pride, had a really bad or stressful day, or because you're just showing off, but whatever the cause, it's definitely not helping. You can disagree with me all you want, but don't insult me by dismissing me as a lesser person than you only to claim everything I said was wrong without providing so much as a single shred of evidence to back it up. You held me to a higher standard on your blog, so I'm going to hold you to that same standard here.

I was the first person to bring numbers there and I'm the first person to bring them here. I've shown you mine, now you show me yours.

  • 2 votes
#1.13 - Fri May 30, 2008 6:24 PM EDT
Danny McGee

Mike and Paul:

1. Please remember the Code of Conduct. Paul, you blatantly insulted the Newsvine team's work so I can understand the heated response from Mike, but Mike, you're a member of the Newsvine team so please set a good example by not responding in kind.

2. As much as I'd love to see the comment count on this article artificially inflated to the point of possibly raising page views, I'd appreciate it if this debate occurred elsewhere, especially if it's going to be an antagonistic one.

3. For what it's worth, I've run this site on much, much slower systems than this and have never had any slow-downs or major stability problems before or since that article. The extreme comment count is the only variable I see that's unique to that article, and that suggests a causal correlation.

  • 3 votes
#1.14 - Fri May 30, 2008 9:06 PM EDT
spiffie

The extreme comment count is the only variable I see that's unique to that article, and that suggests a causal correlation.

I'm pretty sure it is directly related to the number of comments. When I look at the Activity window in Safari, this thread is running about 110 KB right now. However, the article you link to above is running about 2.1 MB. That's a pretty substantial difference, and even over broadband, downloading 2 MB and rendering it in a browser is not instantaneous.

  • 2 votes
#1.15 - Fri May 30, 2008 9:42 PM EDT
Paul William Tenny

2. As much as I'd love to see the comment count on this article artificially inflated to the point of possibly raising page views, I'd appreciate it if this debate occurred elsewhere, especially if it's going to be an antagonistic one.

I'll go ahead and de-track this, then. Sorry for making a mess Danny.

  • 2 votes
#1.16 - Fri May 30, 2008 10:20 PM EDT
Danny McGee

Sorry for making a mess Danny.

No problem. @!$%# happens. :P

  • 2 votes
#1.17 - Fri May 30, 2008 10:29 PM EDT
Mike D.

Danny and spiffie: Yes, you are both exactly correct, and Paul, not surprisingly has changed the subject to trivial CPU differences between browsing sites with and without javascript.

Apologies for calling Paul out, but I frankly have zero tolerance for the large gap between what Paul knows about being a developer and what Paul *thinks* he knows as a developer... at least when it manifests itself as him insulting Newsvine engineers in a public forum.

  • 3 votes
#1.18 - Fri May 30, 2008 11:04 PM EDT
Mike D.

Adam: Yes, absolutely. The jumpable "new" buttons, jumpable stars, and the conversation tracker in general are things no one else on the internet does to my knowledge, and our pagination solution is specifically designed not to break these things. Furthermore, unlike many major news sites which will paginate 1000 comments into 100 pages of 10 each (to juice pageviews), Newsvine pagination's goal is the opposite: essentially, paginate into as few pages as possible... with probably a few hundred or so comments per page. The less disruption the better.

  • 4 votes
#1.19 - Fri May 30, 2008 11:46 PM EDT
Adam Hobson

Furthermore, unlike many major news sites which will paginate 1000 comments into 100 pages of 10 each (to juice pageviews), Newsvine pagination's goal is the opposite: essentially, paginate into as few pages as possible...

I wouldn't have expected anything less from you.

  • 2 votes
#1.20 - Sat May 31, 2008 12:00 AM EDT
Reply
nearing

Okay, I am with you Mr. McGee. that was s-l-o-w.

Multiple pages for those with tons of comments, but something like over 300 or something. We don't want to go crazy with pages.

  • 5 votes
Reply#2 - Fri May 30, 2008 12:44 AM EDT
Mike D.

Yep, see my response to Adam above. The goal is to only break pages when necessary (a few hundred comments apiece is probably the sweet spot).

  • 3 votes
#2.1 - Fri May 30, 2008 11:47 PM EDT
Reply
spiffie

Safari doesn't have much trouble with high-comment counts, thankfully. I guess I could see doing multiple pages if there were a lot of comments (say 500 or 750), but I would hate it if it were more than that. Clicking through pages is really annoying. Another consideration would be how would the tracker react to multiple pages. I'm sure it could be worked out, but it's something to think about. Something else, how would the "Next" links work?

  • 6 votes
Reply#3 - Fri May 30, 2008 1:42 AM EDT
Danny McGee

I guess I could see doing multiple pages if there were a lot of comments (say 500 or 750), but I would hate it if it were more than that. Clicking through pages is really annoying.

It could be an adjustable preference setting, like on vBulletin, et al. I could see the need to page over for more than, say, 100 comments on an older system.

Something else, how would the "Next" links work?

It doesn't seem like it would be that difficult to handle. There could be a standard "Next" link that would load up the next page of comments underneath the main article content, and then a "New" link next to that which would automatically take you to the first New comment on the next page. I'd be more concerned about how to handle sub-threads. You wouldn't want a sub-thread getting cut in half with the second half moved to the next page, for example. But if an early comment gets lots of sub-replies, the code would have to be "smart" enough to move comments older than the new sub-replies to the next page.

  • 2 votes
#3.1 - Fri May 30, 2008 9:13 PM EDT
Mike D.

The next buttons are an issue to deal with, but the hardest part philosophically is deciding whether you break by total number of comments or by total number of master threads. Although it means the exact number of comments per page will not be perfectly equal, our stance is that breaking up by number of master threads is the least troublesome. That way, at least comments never jump from page to page as more come in.

  • 3 votes
#3.2 - Fri May 30, 2008 11:50 PM EDT
spiffie

Is it possible to repaginate dynamically? That is, if the total number of comments on a page grows bigger than X, move the last however many master threads needed to a new page in order to reduce the total number of comments back below X? That way as each master thread continues to grow, a page will always hold about X number of total comments.

  • 1 vote
#3.3 - Sat May 31, 2008 1:38 AM EDT
Mike D.

Yes, that's more or less how it works. But the downside -- if you can call it one -- is that let's say there are 1000 comments. Page 1 might have 325, page 2 might have 400, and page 3 might have 375. It's a downside in that it doesn't necessarily fit people's preconception of pagination in that there is no constant number of comments per page, but it's *not* a downside in that it doesn't really *matter* if that number is constant. This sort of scheme would work very well for 99% of situations, as usually when you have 1000 comments, most are top-level comments and thus they'd break up cleanly... BUT, in the odd case where you have one thread in which two people go back and forth for 500 comments each (never really happens, but it could), the system doesn't really trap for that. That's probably ok though.

  • 2 votes
#3.4 - Sat May 31, 2008 2:53 AM EDT
spiffie

Got it. Can't wait for it to roll out to see what it looks like.

Thanks.

  • 1 vote
#3.5 - Sat May 31, 2008 4:10 AM EDT
Reply
Mike Sifeldeen

I always thought the multiple-page article implementation by websites was used mainly to increase ad-time.

  • 4 votes
Reply#4 - Fri May 30, 2008 2:15 AM EDT
Mike D.

It usually is. :)

  • 2 votes
#4.1 - Fri May 30, 2008 11:50 PM EDT
Reply
foufga

I just tried the page, and it works fine in Firefox 3 RC 1. That said, I do think the comments need to be improved. Below are a couple of 2:23-in-the-AM random thoughts.

What about the slashdot/digg conception of a comment system, where the user can filter through comments based on vote counts--but expands them if needs be)? Also, what about losing the insto-preview in favor of a "preview" button? (As an aside, my preview is frequently rendered incorrectly.) I agree with Paul, 12+ script refs in one page seems excessive.

We could probably lose the Post Comment & Vote button, and probably Check Spelling, no?

I think another thing is that people should use groups more to localize certain discussions. Maybe that's just me.

  • 2 votes
Reply#5 - Fri May 30, 2008 2:21 AM EDT
Allan Neal

Let's fix the tracker first. Safari handled it well, btw. I'm number 1201!

  • 1 vote
Reply#6 - Fri May 30, 2008 2:35 AM EDT
Emily

Hey Danny,

I past your suggestion on the the tech and design guys. They said they had been thinking about ways to address long threads as well.

The time frame for actual implementation is up to them - they're pretty busy and have quite a few projects they're working on.

But I just wanted to let you know I forwarded on your article and the initial response was positive.

  • 5 votes
Reply#7 - Fri May 30, 2008 12:56 PM EDT
Danny McGee

Thanks for the reply, Emily. I'm not too worried about a timeframe, just wanted to make sure you guys were aware of the issue. It probably didn't warrant an article, but I write bug reports twelve hours a day for a living and I haven't written an article in a while, so I figured I'd practice my creative writing skills instead of going through the normal channels. :P

  • 1 vote
#7.1 - Fri May 30, 2008 9:17 PM EDT
Reply
Leave a Comment:
You're in Easy Mode. If you prefer, you can use XHTML Mode instead.
You're in XHTML Mode. If you prefer, you can use Easy Mode instead.
(XHTML tags allowed - a,b,blockquote,br,code,dd,dl,dt,del,em,h2,h3,h4,i,ins,li,ol,p,pre,q,strong,ul)
Newsvine Privacy Statement
As a new user, you may notice a few temporary content restrictions. Click here for more info.
FUN STUFF:
  • Leaderboard |
  • E-Mail Alerts |
  • Top of the Vine |
  • Newsvine Live |
  • Newsvine Archives |
  • The Greenhouse
COMPANY STUFF:
  • Code of Honor |
  • Company Info |
  • Contact Us |
  • Jobs |
  • User Agreement |
  • Privacy Policy |
  • About our ads
LEGAL STUFF:
  • © 2005-2012 Newsvine, Inc. |
  • Newsvine® is a registered trademark of Newsvine, Inc. |
  • Newsvine is a property of msnbc.com