Previous Entry Share Next Entry
Syncitems
morgan
quirrc wrote in ljwin32_sema
If was found that sync in the new version of Semagic (1.3.6) significantly affected performance of LiveJournal.
If you have downloaded it please stop using sync and mass-downloading entries for a while.
The protocol has been disabled because of this right now.
Version 1.3.3.2 is OK
(because it does not have this feature, as well as all other features in the new one are ok).
The problem is that due to large amount of Semagic users (in comparison with other clients also having these archiving features) LiveJournal serveres cannot handle such value of simultaneous downloads.

More info here:
http://www.livejournal.com/community/lj_maintenance/68692.html

http://www.livejournal.com/community/lj_maintenance/68374.html

According to the latest stats at sourceforge( which updates with 2 days delay), there were 2,657 downloads in the first day. That means that almost all viewers of ljwin32_sema and lj_nifty (where it was also posted about cool search feature) downloaded the client specifically to use its new feature and began to download their whole journals from the beginning. Post-per-day count at LJ is currently about 200,000, while (2000 users in one day)*(from 1000 up to 3000) posts = from 10 to 30 times higher.
200,000/24=8,300 posts per hour. This means that 3-5 users downloading journals from the beginning in one hour can double LJ load. 500,000 active users /365 days=1,400. This means that with this load (like 5-6 Oct) it will take a year for all users to download their journals.

  • 1
I don't know what the sync is. I have version 1.3.3.2U. Should I stop using that?

The issue isn't the entire client, just a feature in the new version released just a few days ago.

Posting is fine. The problematic issue (also explained here) is that in the new version, if you view the history, there's an option to synchronize by downloading a copy of all your LJ posts to your computer. This is the specific feature that should NOT be used until further notice -- announcements will likely be either here or in lj_maintenance.

(Deleted comment)
that newest version can't work, but i hope the protocol will be enabled so it won't affect logjam users.

yes but trying to use it will still slow the site down some, not a lot but still can :(

I really liked the feature, but it comes as no surprise that it's a performance drain--that should have been predicted. In order to prevent pissing off LJ developers further, maybe consult with them more in the future?

i've been told the same by avva today, also i was unaware of the load statistics, maybe it should be more open to others.
actually they still don't know (stats broken by high load), if it was because syncitems itself (i.e. large amount of users) or the possiblity to bypass syncitems errors used by many users.

It seems that a significant reason for the client trying to bypass the syncitems errors was from users that posted in more than one language (or in non-Latin languages) before Unicode was implemented, or who don't have the proper "auto-translate from..." property set in their userinfo.

Perhaps if the client could just download the entries without worrying about the encoding...? Or is that impossible?

it gets entries by 50-100 item parts, if there's error in one entry in cannot skip it. So i allowed to bypass it to continue loading. To my mind the problem is that there were just too much users, not in this bypass. It's good that it was stopped now, or otherwise (if there were no such option) they just converted older entries and downloaded them anyway just with a little delay

I am bummed by all this.

I am a huge fan of the download and thought the way it was implemented was wonderful from a user standpoint. Having my journal in one big HTM page is perfect. (I grabbed my download on Sunday and used it to quickly and easily find all of the bad links creating broken images and fix them. It would have taken me forever nearly any other way. And I can search it more quickly than I can type in the search terms!)

At least we still have the new spell check function which totally rocks.

was it clear for you how to get dictionaries?

Yep! Very clear and easy. And it is sooo very fast and functional!

The spellcheck seems to be working the same way for me that it always has, complete with having to hit retry to get the spellcheck to finish.

Did I miss something?

you didnt read 1st my comment to the last post

I read that, but I can't find any test version on the Sourceforge site. Under 1.3 Test, it says "No releases".
Is that spellchecker going to be included in future Semagic releases? Because that's the only feature I've wanted personally. I like spellchecking things before I send them, if I feel I've made mistakes. It's a habit.

I tried out the Sync function. Some of my entries couldn't be downloaded due to encoding issues (error message). I didn't finish downloading, though, since I only wanted to try it out. The bad part was, there wasn't an option in Semagic, that I found, to edit or view the files downloaded to my HD. They're slj files, which I assume can be viewed from within the latest version of Semagic (the previous stable version, which I installed to avoid problems with LJ, doesn't seem to associate with them by default). But I couldn't find an option within Semagic to view or edit them.
I think it would be a good option, because the LJ server can be buggy. And I'm not one to try ten times just to get an entry to be loaded :(
If you can even get the feature to work without taking Livejournal down. Which I'm sure is possible.

Guess I should upgrade soon..... still using 1.2.1.1U I think....

Just wondering if you approved About My Life to use Semagic:
http://www.aboutmylife.net/download/

Not trolling for arguments / dramas, just a little miffed to see my favorite client on another Journal site.
I know that there are specific things that pertain to the legalities of using code, but ... yeah.

*wanders off*

it's opensource and free to use by anyone

hey, i figured i ought to start of with saying how swell it is that semagic is free and useful and cool and all that...

but my problem is with my new beta version of office XP 2003; the spell check doesn't work.. i dont suspect you'll bother to fix this till it goes public, but i thought you might want to know its comin' up.

(Deleted comment)

Russian request for enchancement

Мне охота заявить протест по поводу исчезновения (в версии 1.3.1) правой гипертекстовой панели. Она была удобна, поскольку не только содержала полезную информацию, но и позволяла по вкусу контролировать ширину того «чистого листа», в котором происходит набор текста в дневник. Если эту полезную возможность вернуть, да ещё и позволить пользователю самостоятельно помещать на панель какую угодно информацию по собственному вкусу, то там можно было бы расположить целый предустановленный многостраничный мини-вебсайт: на одной странице клавиатурные сочетания, на другой — ссылки на регулярно посещаемые вебстраницы (в том числе LJ secret love etc.), на третьей — собственные его макросы, и всё это переключается наподобие закладок в диалоге.

Re: Russian request for enchancement

завтра (?)

Hi

Is there any word on when this feature will reappear? It was quite useful for the time it existed. From what I have read, it sounds as if the problem was simply with the way the feature was implimented with the client, will you rewrite and release another client with this feature soon? I see new versions comming out but no word on this?

u read those stats in the post? it's not the client

The two posts linked above suggest the problem isn't the amount of users downloading but the amount of users who were using a client that was 'broken' and caused more load then a client that correctly followed the protocol. Are you saying that even if you can write the feature to work as LJ intended that the server load would still be too much for them?

Is this feature back in? I need the ability to have my journal available offline and preferably with comment scraping as well. I have just downloaded v 1.4.1 OU .

  • 1
?

Log in

No account? Create an account