:)
Commit | Description |
---|---|
Sat Nov 26 14:40:33 CET 2011 by orbiter | - enhanced description on search front page - fixed language and heuristic modifier - added hint to crawl start that we can do also ftp and smb crawls - added a protocol extension to remote crawls to transport all search modifiers to remote peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8108 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartExpert_p.html, htroot/CrawlStartSite_p.html, htroot/index.html, htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/document/ImageParser.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java |
Fri Nov 25 18:33:03 CET 2011 by apfelmaennchen | smaller fixes for YMarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8105 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_metadata.xml, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkEntry.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkUtil.java |
Fri Nov 25 15:09:25 CET 2011 by orbiter | - tested the ARC methods - removed strict authentication (if password is empty; this was buggy and not useful; can be switched on if necessary globally and not for each interface method) - increased speed of CrawlResults page (no dns lookup any more) - increased speed of favicon display (removed dns lookup) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8104 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/Collage.java, htroot/ConfigParser.java, htroot/CrawlResults.html, htroot/CrawlResults.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/YMarks.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/feed.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/termlist_p.java, htroot/api/yacydoc.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_tags.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/manage_tags.java, htroot/api/ynetSearch.java, htroot/index.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/HashARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java |
Fri Nov 25 13:37:00 CET 2011 by orbiter | updated ranking tables (fresh computation) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8103 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: ranking/YBR/YBR-4-00.idx, ranking/YBR/YBR-4-01.idx, ranking/YBR/YBR-4-02.idx, ranking/YBR/YBR-4-03.idx, ranking/YBR/YBR-4-04.idx, ranking/YBR/YBR-4-05.idx, ranking/YBR/YBR-4-06.idx, ranking/YBR/YBR-4-07.idx, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/ranking/BlockRank.java |
Fri Nov 25 12:23:52 CET 2011 by orbiter | some last-minute performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: debian/copyright, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigPortal.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImportOAIPMH_p.java, htroot/PerformanceMemory_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Table_YMark_p.java, htroot/ViewFile.java, htroot/YaCySearchPluginFF.java, htroot/api/getpageinfo_p.java, htroot/api/ymarks/import_ymark.java, htroot/autoconfig.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/sharedBlacklist_p.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCore.java, source/de/anomic/tools/nxTools.java, source/de/anomic/tools/whois.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrShardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/logging/ThreadDump.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/Seed.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/query/RWIProcess.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/visualization/GraphPlotter.java, source/org/apache/tools/tar/TarEntry.java |
Thu Nov 24 19:21:58 CET 2011 by sixcooler | bring my master to stuff from remote Changed Files: .gitignore, build.properties, build.xml, defaults/yacy.network.freeworld.unit, htroot/Status.html, source/de/anomic/data/URLAnalysis.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/MemoryTracker.java, source/net/yacy/peers/graphics/ProfilingGraph.java, startYACY.sh, yacy.logging, yacy.stopwords |
Wed Nov 23 00:50:49 CET 2011 by apfelmaennchen | - included YMarks in addition to old bookmarks in yacysearchitem.html; don't get confused by the old bookmark dialog, the ymark is automatically added silently beforehand. - reworked bookmark creation on crawlstart - many smaller adjustments to ymarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8072 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/api/ymarks/add_ymark.html, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/add_ymark.xml, htroot/api/ymarks/import_ymark.html, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/import_ymark.xml, htroot/api/ymarks/manage_tags.java, htroot/api/ymarks/manage_tags.xml, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks.js, htroot/yacysearchitem.html, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/data/ymark/YMarkTables.java, source/net/yacy/document/Document.java |
Tue Nov 22 13:31:07 CET 2011 by orbiter | added a full, complete, database insert, update and delete API for the tables. Please see this example: list all database tables: http://localhost:8090/api/table_p.xml now create a new table and insert some values into 'mytable' http://localhost:8090/api/table_p.xml?table=mytable&pk=&commitrow=&col_termin=Release%20Machen&col_datum=24.11.2011&col_status=ongoing list the table content: http://localhost:8090/api/table_p.xml?table=mytable&pk= update the table and change a single value inside. You must refer to the row using a primary key 'pk' http://localhost:8090/api/table_p.xml?table=mytable&pk=000000000001&commitrow=&col_datum=29.11.2011 you can also select rows using a search operator http://localhost:8090/api/table_p.xml?table=mytable&pk=&count=10&search= now lets delete the row: http://localhost:8090/api/table_p.xml?table=mytable&pk=&deleterows=pk_000000000001 and we can also delete the complete table: http://localhost:8090/api/table_p.xml?table=mytable&deletetable= You can use this to administrate the robots, bookmarks and API steering using an outside application! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8071 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java, htroot/Tables_p.html, htroot/Tables_p.java, htroot/api/table_p.java, htroot/api/table_p.xml, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java |
Sun Nov 20 23:42:15 CET 2011 by apfelmaennchen | YMarks: - introduced tag manager - a quite powerful tool (still not 100% stable, so be careful) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8060 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/api/ymarks/replace_tags.java, htroot/env/yacy-ymarks.css, htroot/jquery/css/jquery.multiselect.css, htroot/jquery/css/jquery.multiselect.filter.css, htroot/jquery/js/jquery.multiselect.filter.min.js, htroot/jquery/js/jquery.multiselect.min.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkUtil.java |
Sat Nov 19 21:59:21 CET 2011 by apfelmaennchen | YMarks: - added backend and api for tag management git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8058 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/add_ymark.xml, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/replace_tags.java, htroot/api/ymarks/replace_tags.xml, htroot/jquery/css/jquery.multiselect.css, htroot/jquery/js/jquery.multiselect.min.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkUtil.java |
Fri Nov 18 14:09:07 CET 2011 by orbiter | smaller bug fixes for search behavior; should produce less unnecessary removals and an exact number of results as shown in counter should also be a little bit faster git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8057 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigPortal.java, htroot/yacysearch.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/RWIProcess.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/query/SnippetProcess.java |
Fri Nov 18 00:11:27 CET 2011 by apfelmaennchen | YMarks: - show active/running crawls - execute crawls (works currently only if API entry is available) - various smaller fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8056 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/env/yacy-ymarks.css, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkUtil.java |
Tue Nov 15 22:55:00 CET 2011 by apfelmaennchen | - flexigrid images didn't load last time git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8041 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/jquery/flexigrid/css/images/bg.gif, htroot/jquery/flexigrid/css/images/btn-sprite.gif, htroot/jquery/flexigrid/css/images/ddn.png, htroot/jquery/flexigrid/css/images/dn.png, htroot/jquery/flexigrid/css/images/fhbg.gif, htroot/jquery/flexigrid/css/images/first.gif, htroot/jquery/flexigrid/css/images/hl.png, htroot/jquery/flexigrid/css/images/last.gif, htroot/jquery/flexigrid/css/images/line.gif, htroot/jquery/flexigrid/css/images/load.gif, htroot/jquery/flexigrid/css/images/load.png, htroot/jquery/flexigrid/css/images/magnifier.png, htroot/jquery/flexigrid/css/images/next.gif, htroot/jquery/flexigrid/css/images/prev.gif, htroot/jquery/flexigrid/css/images/up.png, htroot/jquery/flexigrid/css/images/uup.png, htroot/jquery/flexigrid/css/images/wbg.gif |
Tue Nov 15 22:45:17 CET 2011 by apfelmaennchen | - updated to jquery flexigrid 1.1 - YMarks.html automatically recognizes if a bookmark is a crawl start git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8040 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/jquery/flexigrid/css/flexigrid.pack.css, htroot/jquery/flexigrid/js/flexigrid.pack.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkDate.java |
Mon Nov 14 23:00:44 CET 2011 by apfelmaennchen | - bugfix for correctly sorting ymarks - some tuning for the autotagger (still not perfect) - /api/ymarks/get_metadata.xml now provides info for crawlstarts - removed unused code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8036 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_metadata.xml, htroot/api/ymarks/get_treeview.java, source/de/anomic/data/ymark/TablesRowComparator.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/data/ymark/YMarkTables.java |
Mon Nov 14 00:19:47 CET 2011 by apfelmaennchen | - some refactoring (ymarks) - improvement for autotagger (is now able to create/detect multi word tags e.g. 'open source') git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8031 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_metadata.xml, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/portalsearch/portalsearch-dev.js, htroot/portalsearch/yacy-portalsearch.js, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java |
Wed Nov 09 22:01:38 CET 2011 by apfelmaennchen | - created new target 'portalsearch' in build.xml to generate yacy-portalsearch.tar.gz for static hosting - some refactoring for search widget and jquery - update for ConfigLiveSearch.html to refelct latest changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8023 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml, htroot/ConfigLiveSearch.html, htroot/YMarks.html, htroot/env/templates/portalheader.template, htroot/env/yacy-ymarks.css, htroot/portalsearch/yacy-portalsearch.css, htroot/portalsearch/yacy-portalsearch.html, htroot/portalsearch/yacy-portalsearch.js |
Wed Nov 09 15:42:55 CET 2011 by orbiter | replaced String with StringBuilder in suggestion process git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8020 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewFile.java, htroot/suggest.java, htroot/yacysearch.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/server/serverObjects.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/cora/ranking/ClusteredScoreMap.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/StringBuilderComparator.java, source/net/yacy/document/WordCache.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/Localization.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/search/query/AccessTracker.java |
Tue Nov 08 12:49:04 CET 2011 by orbiter | - added a new retry connector for solr (for cases where solr responses are slow) - added a new exist property into the metadataRepository which includes solr entries git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8016 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexFederated_p.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrRetryConnector.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/peers/dht/Transmission.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java |
Mon Nov 07 21:44:58 CET 2011 by apfelmaennchen | - update to jquery 1.7 (does not apply to all jquery code, old version is additionally kept for compatibility) - update to jquery-ui 1.8.16 (includes themes) - introduced new portalsearch (as default) - old portalsearch is still available and accessible, but will eventually be removed - jquery and portal search is now loaded by special header templates for maintenance reasons - update to new autocomplete, solves bug: http://bugs.yacy.net/view.php?id=29 - many improvements to YMarks GUI and API...more to come anytime soon Sorry, this is a rather large commit, I hope it doesn't break anything essential, but I need to consolidate some of my efforts in order to move ahead. Especially the update to the portalsearch widget might not be welcomed, but the old one is simply incompatible with newer jquery and jquery-ui libraries, sorry. The code tree /yacy/ui/... is obsolete and will be removed in the future. At that point all productive portalsearches should have migrated to the new version. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8014 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/api/ymarks/import_ymark.java, htroot/env/templates/header.template, htroot/env/templates/jqueryheader.template, htroot/env/templates/metas.template, htroot/env/templates/portalheader.template, htroot/env/templates/portalinput.template, htroot/index.html, htroot/jquery/css/jquery-ui-combobox.css, htroot/jquery/css/yacy-portalsearch.css, htroot/jquery/css/yacy-ymarks.css, htroot/jquery/js/jquery-1.7.min.js, htroot/jquery/js/jquery-ui-1.8.16.custom.min.js, htroot/jquery/js/jquery-ui-combobox.js, htroot/jquery/js/jquery.field-0.9.2.min.js, htroot/jquery/js/jquery.form-2.73.js, htroot/jquery/js/jquery.query-2.1.7.js, htroot/jquery/js/yacy-portalsearch.js, htroot/jquery/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/jquery/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png, htroot/jquery/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png, htroot/jquery/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png, htroot/jquery/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png, htroot/jquery/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png, htroot/jquery/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/jquery/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png, htroot/jquery/themes/smoothness/images/ui-icons_222222_256x240.png, htroot/jquery/themes/smoothness/images/ui-icons_2e83ff_256x240.png, htroot/jquery/themes/smoothness/images/ui-icons_454545_256x240.png, htroot/jquery/themes/smoothness/images/ui-icons_888888_256x240.png, htroot/jquery/themes/smoothness/images/ui-icons_cd0a0a_256x240.png, htroot/jquery/themes/smoothness/jquery-ui-1.8.16.custom.css, htroot/jquery/themes/start/images/ui-bg_flat_55_999999_40x100.png, htroot/jquery/themes/start/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/jquery/themes/start/images/ui-bg_glass_45_0078ae_1x400.png, htroot/jquery/themes/start/images/ui-bg_glass_55_f8da4e_1x400.png, htroot/jquery/themes/start/images/ui-bg_glass_75_79c9ec_1x400.png, htroot/jquery/themes/start/images/ui-bg_gloss-wave_45_e14f1c_500x100.png, htroot/jquery/themes/start/images/ui-bg_gloss-wave_50_6eac2c_500x100.png, htroot/jquery/themes/start/images/ui-bg_gloss-wave_75_2191c0_500x100.png, htroot/jquery/themes/start/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/jquery/themes/start/images/ui-icons_0078ae_256x240.png, htroot/jquery/themes/start/images/ui-icons_056b93_256x240.png, htroot/jquery/themes/start/images/ui-icons_d8e7f3_256x240.png, htroot/jquery/themes/start/images/ui-icons_e0fdff_256x240.png, htroot/jquery/themes/start/images/ui-icons_f5e175_256x240.png, htroot/jquery/themes/start/images/ui-icons_f7a50d_256x240.png, htroot/jquery/themes/start/images/ui-icons_fcd113_256x240.png, htroot/jquery/themes/start/jquery-ui-1.8.16.custom.css, htroot/yacy/ui/css/jquery.multiselect.css, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery.multiselect.min.js, htroot/yacy/ui/sidebar/sidebar_1.html |
Tue Oct 04 11:06:24 CEST 2011 by orbiter | refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7987 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistImpExp_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigBasic.java, htroot/ConfigProfile_p.java, htroot/Connections_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/RemoteCrawl_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewProfile.java, htroot/Wiki.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/feed.java, htroot/api/queues_p.java, htroot/api/timeline.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/idx.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/net/yacy/peers/Accessible.java, source/net/yacy/peers/EventChannel.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/NewsDB.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/NewsQueue.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/dht/Dispatcher.java, source/net/yacy/peers/dht/FlatWordPartitionScheme.java, source/net/yacy/peers/dht/PartitionScheme.java, source/net/yacy/peers/dht/PeerSelection.java, source/net/yacy/peers/dht/Transmission.java, source/net/yacy/peers/dht/VerticalWordPartitionScheme.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/ranking/BlockRank.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/yacy.java |
Fri Sep 30 14:38:28 CEST 2011 by orbiter | added directDocByURL attribute in crawl profile git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7985 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/CrawlStartExpert_p.html, htroot/CrawlStartExpert_p.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/search/Switchboard.java |
Fri Sep 30 10:26:31 CEST 2011 by orbiter | another performance hack: re-use of known host addresses for isLocal property; avoids look-up in local hash git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7983 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Collage.java, htroot/yacy/search.java, source/de/anomic/crawler/Latency.java, source/de/anomic/server/serverCore.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/peers/yacySeed.java |
Fri Sep 30 09:57:50 CEST 2011 by orbiter | - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill) - this may have also (good) performance side effects on other parts of YaCy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7982 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Supporter.java, htroot/Surftips.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/peers/yacyNewsDB.java, source/net/yacy/peers/yacyNewsQueue.java, source/net/yacy/search/index/MetadataRepository.java |
Thu Sep 29 17:17:39 CEST 2011 by orbiter | implemented crawl restrictions for IP pattern and country lists git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7980 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java, htroot/CrawlStartExpert_p.html, htroot/CrawlStartExpert_p.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/search/Switchboard.java |
Tue Sep 27 23:58:18 CEST 2011 by orbiter | added crawl settings for three new filters for each crawl: must-match for IPs (IPs that are known after DNS resolving for each URL in the crawl queue) must-not-match for IPs must-match against a list of country codes (allows only loading from hosts that are hostet in given countries) note: the settings and input environment is there with that commit, but the values are not yet evaluated git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7976 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartExpert_p.html, htroot/CrawlStartExpert_p.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java |
Mon Sep 26 23:42:28 CEST 2011 by orbiter | refactoring: RankingProcess -> RWIProcess ResultFetcher -> SnippetProcess git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7974 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, htroot/IndexControlRWIs_p.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/peers/yacyClient.java, source/net/yacy/peers/yacySearch.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/RWIProcess.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SnippetProcess.java |
Sun Sep 25 18:59:06 CEST 2011 by orbiter | refactoring: - moved many classes from de.anomic to net.yacy - made more sub-packages for search classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7973 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml, htroot/AccessPicture_p.java, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigHTCache_p.java, htroot/ConfigHeuristics_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexFederated_p.java, htroot/IndexImportMediawiki_p.java, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/RemoteCrawl_p.java, htroot/SearchEventPicture.java, htroot/ServerScannerList.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Table_API_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewProfile.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/YMarks.java, htroot/YaCySearchPluginFF.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/feed.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/table_p.java, htroot/api/termlist_p.java, htroot/api/timeline.java, htroot/api/trail_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/ynetSearch.java, htroot/api/version.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_tags.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/index.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/suggest.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/idx.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, htroot/yacysearchlatestinfo.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/ListManager.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/RobotsTxtConfig.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/UPnP.java, source/de/anomic/tools/whois.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/gui/Tray.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/migration.java, source/net/yacy/peers/dht/Dispatcher.java, source/net/yacy/peers/dht/FlatWordPartitionScheme.java, source/net/yacy/peers/dht/PartitionScheme.java, source/net/yacy/peers/dht/PeerSelection.java, source/net/yacy/peers/dht/Transmission.java, source/net/yacy/peers/dht/VerticalWordPartitionScheme.java, source/net/yacy/peers/graphics/EncodedImage.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/peers/graphics/OSMTile.java, source/net/yacy/peers/graphics/ProfilingGraph.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/peers/operation/yacyBuildProperties.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacySeedUploadFile.java, source/net/yacy/peers/operation/yacySeedUploadFtp.java, source/net/yacy/peers/operation/yacySeedUploadScp.java, source/net/yacy/peers/operation/yacySeedUploadScp.xml, source/net/yacy/peers/operation/yacySeedUploader.java, source/net/yacy/peers/operation/yacyUpdateLocation.java, source/net/yacy/peers/operation/yacyVersion.java, source/net/yacy/peers/yacyAccessible.java, source/net/yacy/peers/yacyChannel.java, source/net/yacy/peers/yacyClient.java, source/net/yacy/peers/yacyCore.java, source/net/yacy/peers/yacyNetwork.java, source/net/yacy/peers/yacyNewsDB.java, source/net/yacy/peers/yacyNewsPool.java, source/net/yacy/peers/yacyNewsQueue.java, source/net/yacy/peers/yacyPeerActions.java, source/net/yacy/peers/yacySearch.java, source/net/yacy/peers/yacySeed.java, source/net/yacy/peers/yacySeedDB.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/IndexAbstracts.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/index/Segments.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/ResultFetcher.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/ranking/BlockRank.java, source/net/yacy/search/ranking/RankingProcess.java, source/net/yacy/search/ranking/RankingProfile.java, source/net/yacy/search/ranking/ReferenceOrder.java, source/net/yacy/search/snippet/ContentDomain.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/yacy.java, test/de/anomic/yacy/yacyVersionTest.java |
Thu Sep 15 12:01:21 CEST 2011 by orbiter | added a concurrent warmin-up of Table data structures. that should speed-up the start-up process but may also cause stronger CPU load at that time. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7956 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java |
Wed Sep 14 11:59:30 CEST 2011 by orbiter | moved the log, memory, processes and the messages into a new computation monitor main menu item git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7951 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/News.html, htroot/PerformanceConcurrency_p.html, htroot/PerformanceMemory_p.html, htroot/PerformanceSearch_p.html, htroot/Settings_p.html, htroot/ViewLog_p.html, htroot/env/base.css, htroot/env/templates/header.template, htroot/env/templates/submenuAccessTracker.template, htroot/env/templates/submenuComputation.template |
Wed Sep 14 03:24:56 CEST 2011 by orbiter | redesign and simplification of main menu; bundling of some sub-menues git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7948 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigAppearance_p.html, htroot/ConfigLanguage_p.html, htroot/ConfigProfile_p.html, htroot/CrawlResults.html, htroot/Ranking_p.html, htroot/Threaddump_p.html, htroot/ViewLog_p.html, htroot/env/base.css, htroot/env/templates/header.template, htroot/env/templates/submenuAccessTracker.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCrawlMonitor.template, htroot/env/templates/submenuPortalIntegration.template |
Tue Sep 13 16:39:41 CEST 2011 by orbiter | YaCy can now use the solr index to compute text snippets. This makes search result preparation MUCH faster because no document fetching and parsing is necessary any more. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7943 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, htroot/IndexFederated_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/document/Document.java |
Wed Sep 07 12:08:57 CEST 2011 by orbiter | added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7931 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/repository/LoaderDispatcher.java, test/de/anomic/document/ParserTest.java |
Thu Sep 01 18:05:00 CEST 2011 by orbiter | - added a 'add every media object linked in a html document as a new document' to the html parser. This causes that all image, app, video or audio file that is linked in a html file is added as document. In fact that means that parsing a single html document may cause that a number of documents is inserted into the search index. - some refactoring for mime type discovery git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7919 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/MediaSnippet.java, source/net/yacy/document/Classification.java, source/net/yacy/document/parser/htmlParser.java |
Sun Aug 28 02:16:36 CEST 2011 by low012 | *) Replaced occurrences of "Wikimedia" with "MediaWiki" where applicable. (Thanks to the folks of 0x20.be for pointing this out.) *) Added description of where to place MediaWiki dump for import. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7905 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/importmediawiki.sh, htroot/CrawlResults.html, htroot/IndexImportMediawiki_p.html, htroot/IndexImportMediawiki_p.java, htroot/env/templates/submenuIndexCreate.template, htroot/mediawiki_p.java, locales/de.lng, source/net/yacy/document/importer/MediawikiImporter.java |
Thu Aug 25 12:35:25 CEST 2011 by orbiter | added new methods to count the number of objects in RWIs. lots of refactoring was necessary to introduce new Rating class and to unify naming of methods git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7896 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, source/de/anomic/search/Segment.java, source/de/anomic/yacy/dht/Dispatcher.java, source/net/yacy/cora/ranking/AbstractOrder.java, source/net/yacy/cora/ranking/Order.java, source/net/yacy/cora/ranking/Rating.java, source/net/yacy/cora/ranking/RatingOrder.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/StringOrder.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/BufferedIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/IndexReader.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceContainerOrder.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/yacy.java |
Tue Aug 23 22:18:30 CEST 2011 by orbiter | refactoring in cora: added sorting package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7890 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/net/yacy/cora/ranking/WeakPriorityBlockingQueue.java, source/net/yacy/cora/services/federated/SearchResult.java, source/net/yacy/cora/sorting/Array.java, source/net/yacy/cora/sorting/Sortable.java, source/net/yacy/cora/sorting/UpDownLatch.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/yacy.java |
Tue Aug 23 00:37:53 CEST 2011 by orbiter | refactoring: moved all score-related classes to new ranking package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7889 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Supporter.java, htroot/Surftips.java, htroot/yacy/search.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/search/BlockRank.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/dht/PeerSelection.java, source/net/yacy/cora/ranking/AbstractScoreMap.java, source/net/yacy/cora/ranking/ClusteredScoreMap.java, source/net/yacy/cora/ranking/ConcurrentScoreMap.java, source/net/yacy/cora/ranking/OrderedScoreMap.java, source/net/yacy/cora/ranking/ReversibleScoreMap.java, source/net/yacy/cora/ranking/ScoreComparator.java, source/net/yacy/cora/ranking/ScoreMap.java, source/net/yacy/cora/services/federated/SearchHub.java, source/net/yacy/document/WordCache.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/yacy.java |
Mon Aug 22 19:50:03 CEST 2011 by sixcooler | Implementation of strategies for controlling memory resources. You can toggle between previous (standard) and new (generation) strategy at PerformanceMemory_p.html. The generation memory strategy is implemented with the objective of running more robust but with the cost of early stopping some tasks (eg. dht) while running low on memory. This new strategy does respect the generational way a heap is organized on most used jvms. These changes run fine on my 3 peers for weeks now, but as I'm human, I may fail. Please be carefull using generation memory strategy and report errors by naming OS, jvm and java_args. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7886 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, source/de/anomic/crawler/ResourceObserver.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/MemoryStrategy.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/yacy.java |
Fri Aug 12 19:42:32 CEST 2011 by sixcooler | bump to httpclient-4.1.2, httpcore-4.1.3 - bugfixrelease git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7876 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/httpclient-4.1.2.License, lib/httpclient-4.1.2.jar, lib/httpcore-4.1.3.License, lib/httpcore-4.1.3.jar, lib/httpmime-4.1.2.License, lib/httpmime-4.1.2.jar |
Wed Aug 03 20:47:43 CEST 2011 by orbiter | performance hack and ensuring termination in serverAccessTracker. cause: "Session_:53600#0_POST /yacy/hello.html HTTP/1.1" prio=10 tid=0x2322b000 nid=0x3ba7 runnable [0x03d3e000] java.lang.Thread.State: RUNNABLE at java.lang.Long.valueOf(Long.java:557) at de.anomic.server.serverAccessTracker.clearTooOldAccess(serverAccessTracker.java:113) at de.anomic.server.serverAccessTracker.cleanupAccessTracker(serverAccessTracker.java:75) - locked <0x3bda2ae8> (a de.anomic.server.serverAccessTracker) at de.anomic.server.serverAccessTracker.track(serverAccessTracker.java:125) at de.anomic.server.serverSwitch.track(serverSwitch.java:542) at de.anomic.http.server.HTTPDemon.parseRequestLine(HTTPDemon.java:641) at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:491) at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at de.anomic.server.serverCore$Session.listen(serverCore.java:757) at de.anomic.server.serverCore$Session.run(serverCore.java:651) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7862 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverAccessTracker.java |
Tue Aug 02 01:28:23 CEST 2011 by sixcooler | stop loading via http at defined maximum of bytes - even size is unknown before loading using max-file-size of type int for parsing documents (since content is used as byte-arrays, 'integer' should be maximum) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7855 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/DictionaryLoader_p.java, htroot/Load_RSS_p.java, htroot/ViewFile.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/OSMTile.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/repository/LoaderDispatcher.java |
Sat Jul 16 12:08:43 CEST 2011 by orbiter | fixed bugs and deadlocks in core database indexing structures: - added new Array class that contains an abstraction of the java Arrrays class which replaces the home-brew quicksort algorithm. - the new class is about four times slower than the old one, but it works correct (the old one had errors) - fixed a synchronization problem git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7842 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Array.java, source/net/yacy/kelondro/order/Sortable.java, source/net/yacy/yacy.java |
Fri Jul 15 10:38:10 CEST 2011 by orbiter | changed handling of RowSet element retrieval: until today all elements had been copied from the underlying byte[] arrays into a new Entry object that again had a copy of a portion of that byte[] in its own bye[]. There was an option to just refer to the underlying byte[] with a pointer but that was almost never used. This commit now changes an interface to the Row class where it is now necessary to tell if a copy is always required. Fortunately the copy is only needed in very rare cases. That means that this change should cause much less memory allocation; it is expected that this happens especially during search situations. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7840 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/yacy/yacyNewsDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java |
Thu Jul 14 09:07:06 CEST 2011 by orbiter | added a visualization of peer pings to the performance graphic git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7837 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.java, htroot/PerformanceSearch_p.java, htroot/yacy/hello.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/PrintTool.java |
Mon Jul 04 18:35:30 CEST 2011 by orbiter | - replaced solrj 3.1 with solrj 3.3 - updated also slf4j - added authentication for solrj git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7829 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, defaults/yacy.init, htroot/IndexFederated_p.java, lib/apache-solr-solrj-3.3.0.License, lib/apache-solr-solrj-3.3.0.jar, lib/slf4j-api-1.6.1.jar, lib/slf4j-jdk14-1.6.1.jar, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrHTTPClient.java |
Mon Jul 04 01:55:55 CEST 2011 by low012 | *) Invalid crawl profiles (containing invalid mustmatch/mustnotmatch filters) will be moved from active crawls to invalid crawls (new file: DATA/INDEX/freeworld/QUEUES/crawlProfilesInvalid.heap). This file can not be edited yet, but it shoudl be easy to extend the CrawlProfileEditor accordingly. *) Corrupt crawlProfilesPassive.heap would cause crawlProfilesActive.heap to be deleted. Don't know if this ever happend, but will not happen anymore. *) Cleaned up a little bit. *) Added some comments. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7827 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/net/yacy/cora/services/federated/yacy/CacheStrategy.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/repository/RegexHelper.java |
Sun Jul 03 23:33:24 CEST 2011 by orbiter | setting startup options: -Xss256k and -XX:ReservedCodeCacheSize=1024m after appearance of a malloc error together with a crash of the jvm which stated at the end of the log: # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 32756 bytes for ChunkPool::allocate # Possible reasons: # The system is out of physical RAM or swap space # In 32 bit mode, the process size limit was hit # Possible solutions: # Reduce memory load on the system # Increase physical memory or swap space # Check if swap backing store is full # Use 64 bit Java on a 64 bit OS # Decrease Java heap size (-Xmx/-Xms) # Decrease number of Java threads # Decrease Java thread stack sizes (-Xss) # Set larger code cache with -XX:ReservedCodeCacheSize= this follows the last two points in the list of recommendations. To set appropriate values the default values from http://www.oracle.com/technetwork/java/hotspotfaq-138619.html and http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html had been considered git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7823 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, startYACY.bat, startYACY.sh |
Sun Jul 03 22:44:33 CEST 2011 by orbiter | fix for: java.lang.NullPointerException at net.yacy.kelondro.index.RowCollection.<init>(RowCollection.java:97) at net.yacy.kelondro.index.RowSet.<init>(RowSet.java:48) at net.yacy.kelondro.rwi.ReferenceContainer.<init>(ReferenceContainer.java:58) at net.yacy.kelondro.rwi.ReferenceIterator.next(ReferenceIterator.java:69) at net.yacy.kelondro.rwi.ReferenceIterator.next(ReferenceIterator.java:43) at net.yacy.kelondro.blob.ArrayStack.merge(ArrayStack.java:1023) at net.yacy.kelondro.blob.ArrayStack.mergeWorker(ArrayStack.java:922) at net.yacy.kelondro.blob.ArrayStack.mergeMount(ArrayStack.java:869) at net.yacy.kelondro.rwi.IODispatcher$MergeJob.merge(IODispatcher.java:267) at net.yacy.kelondro.rwi.IODispatcher$MergeJob.access$300(IODispatcher.java:239) at net.yacy.kelondro.rwi.IODispatcher.run(IODispatcher.java:180) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7822 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/util/LookAheadIterator.java |
Sun Jul 03 00:33:35 CEST 2011 by orbiter | replaced xerces.jar library that was originally added 2005 with SVN 126 to the libx directory and that was moved to lib in SVN 5781 the new replacement is taken from http://xerces.apache.org and has the version 2.11.0 and was inside the file Xerces-J-bin.2.11.0.tar.gz and consists of two files named xercesImpl.jar and xml-apis.jar The original purpose of that library was to support: - content parsers - optional seed uploader - SOAP API (which will be committed later) Since the SOAP API does not exist any more the purpose is to support content parser and an optional seed uploader git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7819 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/xercesImpl.jar, lib/xercesImpl.license, lib/xml-apis.jar, lib/xml-apis.license |
Thu Jun 30 17:49:21 CEST 2011 by orbiter | - added an index profile editor in the /indexFederated_p.html servlet for solr indexes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7811 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, defaults/yacy.init, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/cora/storage/ConfigurationSet.java |
Thu Jun 23 13:57:17 CEST 2011 by orbiter | - added a search option to filter only specific network protocols. i.e. get only results from ftp servers. Just add '/ftp' to your search. for example search for "passwd /ftp". This can also be done with /http /https and /smb - fixed some search throttling processes that should protect your peer against search DoS or strong search load git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7794 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/workflow/AbstractThread.java |
Mon Jun 13 23:44:03 CEST 2011 by orbiter | - more attributes for search progress bar - moved cache strategy to cora package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7778 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Load_RSS_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/util/getpageinfo_p.java, htroot/js/yacysearch.js, htroot/yacy/search.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchlatestinfo.java, htroot/yacysearchlatestinfo.json, htroot/yacysearchtrailer.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/services/federated/SearchHub.java, source/net/yacy/cora/services/federated/SearchResult.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/yacy/CacheStrategy.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/repository/LoaderDispatcher.java |
Thu Jun 02 23:34:50 CEST 2011 by sixcooler | new httpclient, httcore (bugfixrelease) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7769 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/httpclient-4.1.1.License, lib/httpclient-4.1.1.jar, lib/httpcore-4.1.1.License, lib/httpcore-4.1.1.jar, lib/httpmime-4.1.1.License, lib/httpmime-4.1.1.jar |
Wed Jun 01 21:31:56 CEST 2011 by orbiter | - applied many small performance hacks - added a memory limitation in the zip parser and the pdf parser - added a search throttling: if there are too many search queries are still to be computed, then new requests are not accepted for some time. if after a one second still no space is there to perform another search, the search terminates with no results. this case should only happen in case of DoS-like situations and in case of strong load on a peer like if it is integrated in metager. - added a search cache deletion process that removes search requests in case that throttling happens git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7766 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/ListManager.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEventCache.java, source/net/yacy/cora/storage/ConcurrentScoreMap.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/repository/LoaderDispatcher.java |
Fri May 27 10:24:54 CEST 2011 by orbiter | hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources: used a ASCII String <-> byte[] conversion wherever possible. Many Strings in YaCy are hashes which are pure ASCII (base64 hashes). The new ASCII String <-> byte[] conversion method have less computation overhead than the UTF8 conversion. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7746 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/ConfigNetwork_p.java, htroot/CrawlResults.java, htroot/CrawlStartScanner_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/Supporter.java, htroot/ViewFile.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/queues_p.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/yacy/crawlReceipt.java, htroot/yacy/idx.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxtEntry.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/ChunkedOutputStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/BlockRank.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/ASCII.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/dbtest.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java |
Thu May 26 18:34:35 CEST 2011 by orbiter | Replaced a widely-used Property Object in the httpd with HashMap<String, Object> which is not synchronized like Properties A synchronization is not needed here and applies an overhead to the httpd process which is now removed. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7745 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/MimeTable.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/search/ResultFetcher.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/kelondro/util/FileUtils.java |
Thu May 26 12:57:02 CEST 2011 by orbiter | - enhanced ybr ranking computation - many speed/performance hacks - added solr charding and new charding web interface - added option to switch off the yacy index when using solr - added new fail-url categories which are used to make a distinction which fail-urls to be sent to solr - refactoring/renaming of some method names to distinguish host/url hashes better - a large number of bug/npe fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7738 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, htroot/env/templates/submenuIndexControl.template, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/search/BlockRank.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrChardingSelection.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/cora/storage/AbstractScoreMap.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceVars.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/yacy.java |
Mon May 23 23:08:04 CEST 2011 by orbiter | small features, some bug fixes and performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7733 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/Load_RSS_p.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/rwi/IndexCell.java |
Wed May 18 16:26:28 CEST 2011 by orbiter | this is a re-implementation of the YaCy Block Rank feature This time it works like this: - each peer provides its ranking information using the yacy/idx.json servlet - peers with more than 1 GB ram will load this information from all other peers, combine that into one ranking table and store it locally. This happens during the start-up of the peer concurrently. The new generated file with the ranking information is at DATA/INDEX/<network>/QUEUES/hostIndex.blob - this index is then computed to generate a new fresh ranking table. Peers which can calculate their own ranking table will do that every start-up to get latest feature updates until the feature is stable - I computed new ranking tables as part of the distribition and commit it here also - the YBR feature must be enabled manually by setting the YBR value in the ranking servlet to level 15. A default configuration for that is also in the commit but it does not affect your current installation only fresh peers - a recursive block rank refinement is implemented but disabled at this point. it needs more testing Please play around with the ranking settings and see if this helped to make search results better. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7729 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/yacy/idx.java, ranking/YBR/YBR-4-00.idx, ranking/YBR/YBR-4-01.idx, ranking/YBR/YBR-4-02.idx, ranking/YBR/YBR-4-03.idx, ranking/YBR/YBR-4-04.idx, ranking/YBR/YBR-4-05.idx, ranking/YBR/YBR-4-06.idx, ranking/YBR/YBR-4-07.idx, ranking/YBR/YBR-4-08.idx, ranking/YBR/YBR-4-09.idx, ranking/YBR/YBR-4-0a.idx, ranking/YBR/YBR-4-0b.idx, source/de/anomic/http/client/Cache.java, source/de/anomic/search/BlockRank.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyBuildProperties.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/net/yacy/cora/storage/AbstractScoreMap.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/migration.java |
Mon May 16 00:57:31 CEST 2011 by orbiter | added a new yacy protocol servlet 'idx'. This returns an index to one of the data entities that is stored in YaCy. This servlet currently only serves for indexes to the web structure hosts. It can be tested by calling http://localhost:8090/yacy/idx.json?object=host This yacy protocol servlet is the first one that returns JSON code and that also shows index entries in a readable format. This will make the development of API applications much easier. This is also an example implementation for possible json versions of the other existing YaCy protocol interfaces. The main purpose of this new feature is to provide a distributed block rank collection feature. Creating a block rank is very difficult if the forward-link data is first collected and then one peer must create a backward-link index. This interface provides already a partial backward index and therefore a collection of all these indexes needs only to be joined which is very easy. The result should be the computation of new block rank tables that all peers can perform. To reduce load from peers this servlet buffers all data and refreshes it only once in 12 hours. This very slow update cycle is needed because the interface will be called round-robin from all peers once after start-up. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7724 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/api/webstructure.java, htroot/yacy/idx.java, htroot/yacy/idx.json, htroot/yacy/search.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java |
Fri May 13 08:21:40 CEST 2011 by orbiter | code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7713 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/CrawlStartScanner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Settings_p.java, htroot/Table_API_p.java, htroot/ViewFile.java, htroot/api/yacydoc.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/dht/Transmission.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/migration.java, source/net/yacy/yacy.java |
Sat May 07 09:37:46 CEST 2011 by orbiter | reduce teh effect of 'Bildersuche findet generierte HTML-Seiten als Bilder' see http://bugs.yacy.net/view.php?id=9 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7705 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewImage.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/parser/htmlParser.java |
Fri May 06 00:37:06 CEST 2011 by orbiter | - relaxed matching of string-search (this is now case-insensitive) - added transport of string-search pattern to remote search protocol - fixed a problem parsing snippets with a '-' inside git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7700 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigParser.html, htroot/Table_API_p.java, htroot/yacy/search.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/document/Condenser.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java |
Thu May 05 02:25:14 CEST 2011 by orbiter | - added full-String search function: find things that match exactly what is quoted in the query - re-structuring authentification methods to fix a problem with API steering git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7697 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/User.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/migration.java, source/net/yacy/yacy.java |
Mon May 02 16:05:51 CEST 2011 by orbiter | - refactoring of robots - added option to crawler to send error-URLs to solr - changed solr scheme slightly (no multi-value fields where no multi values are) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7693 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/WebStructurePicture_p.java, htroot/api/util/getpageinfo_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/RobotsTxtEntry.java, source/de/anomic/crawler/RobotsTxtParser.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/YMarkEntry.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Sun May 01 23:42:48 CEST 2011 by apfelmaennchen | YMark: - general improvements on importers, especially on auto tagging - added get_tags (needed for tag clouds etc.) - improved flexigrid support - added YMarks.html (not fully working) that will eventually replace Bookmarks.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7691 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/YMarks.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/get_tags.java, htroot/api/ymarks/get_tags.xml, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/test_import.html, htroot/yacy/ui/sidebar/sidebar_2.html, source/de/anomic/data/ymark/TablesRowComparator.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkEntry.java, source/de/anomic/data/ymark/YMarkHTMLImporter.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkTag.java, source/de/anomic/data/ymark/YMarkXBELImporter.java |
Fri Apr 29 12:58:12 CEST 2011 by orbiter | some patches to prevent that empty or bad IP information is broadcasted - on client-side: fix bad IP reports from remote Peers by replacing their reported IP with their server IP if the reported IP is bad, broken or disallowed - on server-side: the same during a peer ping (here the ping'ed server acts also as client during the back-ping) and also when receiving a message or a search where the client sends also its seed. Here the IP is replaced by the client IP if the reported IP is broken or bad git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7687 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml, htroot/Status.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java |
Fri Apr 29 03:52:20 CEST 2011 by orbiter | added a magic to the peer ping that will be used in case that the contacting peer requests that it's reported IP shall be used for a back-ping. The back-ping now also returns the same magic which will make it possible that the requested peer can verify that the back-pinged peer is actually the same peer. This is also a protection against the foced-fake of a external IP: if such an IP was faked, then the next ping from the affected peer to another peer looks like a staticIP report. Such a bad staticIP-by-faked-response can now be discovered and fixed by the peer that gets the second ping after the first ping contained a faked response. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7683 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/hello.java, htroot/yacy/query.html, htroot/yacy/query.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java |
Thu Apr 28 15:09:01 CEST 2011 by orbiter | added more attributes for html parser and enhanced data structures git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7679 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/clearcache.sh, bin/clearindex.sh, bin/importurllist.sh, build.xml, htroot/WebStructurePicture_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/storage/AbstractScoreMap.java, source/net/yacy/cora/storage/ClusteredScoreMap.java, source/net/yacy/cora/storage/ConcurrentScoreMap.java, source/net/yacy/cora/storage/OrderedScoreMap.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/html/Scraper.java, source/net/yacy/document/parser/html/TransformerWriter.java |
Tue Apr 26 15:35:29 CEST 2011 by orbiter | better abstraction of http client identification git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7675 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/Network.java, htroot/sharedBlacklist_p.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/ClientIdentification.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java |
Thu Apr 21 15:58:49 CEST 2011 by orbiter | - enhanced html parser: recognized much more details in the content - added more properties to solr index - refactoring - more constants in switchboard - fix for some NPEs - recognition of more images - removed synchronization in HandleMap (obviously not necessary?) - added a nolocal configuration to remove excessive dns lookup (works only on allip - default off). Indexes produced with this setting are all flagged with 'local' and are (on purpose) not usable for freeworld because they will be rejected as beeing local. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7672 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigNetwork_p.java, htroot/Crawler_p.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/HandleMap.java |
Mon Apr 18 23:42:14 CEST 2011 by apfelmaennchen | YMark: - more refactoring >> YMarkEntry - integration of SurrogateReader as bookmark importer - various small bug fixes e.g. get_xbel.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7668 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_xbel.xml, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/test_import.html, source/de/anomic/data/ymark/YMarkDate.java, source/de/anomic/data/ymark/YMarkEntry.java, source/de/anomic/data/ymark/YMarkHTMLImporter.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkUtil.java, source/de/anomic/data/ymark/YMarkXBELImporter.java |
Mon Apr 18 18:11:16 CEST 2011 by orbiter | - fixed a bug in crawl start with file name (npe in new url) - added deletion of solr index in IndexControlRWIs - added asynchronous adding of large url lists (happens when crawls are startet with file) - fixed npe in Image display - replaced language warning with fine logging - added a domain name cache in Domains that helps to speed up the isLocal property (less DNS lookups) - added a new storage class for this new cache: KeyList. The domain key list is stored in DATA/WORK/globalhosts.list - added concurrent solr updates and chunked transfers (50 documents until a commit is done) for high-speed feeding (> 40000 ppm) - fixed a bug in content scraper that chopped off large parts of crawl lists (using crawl start from file) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7666 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/services/federated/solr/SolrHTTPClient.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/document/parser/html/ContentScraper.java |
Fri Apr 15 22:58:58 CEST 2011 by apfelmaennchen | YMark: - first attempt for a firefox json bookmark importer - added JSON library json-simple-1.1.jar git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7658 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, build.xml, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/test_import.html, lib/json-simple-1.1.License, lib/json-simple-1.1.jar, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkUtil.java, source/de/anomic/data/ymark/YMarkXBELImporter.java |
Fri Apr 15 17:54:19 CEST 2011 by orbiter | enhanced location search: search is now done using verify=false (instead of verify=cacheonly) which will cause that much more targets can be found. This showed a bug where no location information was used from the metadata (and other metadata information) if cache=false is requested. The bug was fixed. Added also location parsing from wikimedia dumps. A wikipedia dump can now also be a source for a location search. Fixed many smaller bugs in connection with location search. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7657 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch_location.html, htroot/yacysearch_location.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/document/Document.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/html/ContentScraper.java |
Thu Apr 14 22:05:04 CEST 2011 by orbiter | Added federated index storage to solr. YaCy supports now the storage to remote solr indexes. More federated storage (and search) methods may follow. The remote index scheme is the same as produced by the SolrCell; see http://wiki.apache.org/solr/ExtractingRequestHandler Because this default scheme is used, the default example scheme can be used as solr configuration This is also the same scheme that solr uses if documents are imported with apache tika. federated solr storage is switched off by default. To use this, do the following: - set federated.service.solr.indexing.enabled = true - download solr from http://www.apache.org/dyn/closer.cgi/lucene/solr/ - extract the solr (3.1) package, 'cd example' and start solr with 'java -jar start.jar' - start yacy and then start a crawler. The crawler will fill both, YaCy and solr indexes. - to check whats in solr after indexing, open http://localhost:8983/solr/admin/ Until now it is not possible to use the solr index to search with YaCy in that solr index. This functionality is now available for two reasons: 1) to compare the functionality of Solr and YaCy and to compare the search speed 2) to use YaCy as a search appliance for people who need a crawler or other source harvesting methods that YaCy provides (like dublin core reading, wikimedia dump reading, rss feed reader etc) if people still want to use solr instead of YaCy. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7654 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, defaults/yacy.init, htroot/Table_YMark_p.java, htroot/api/ymarks/get_ymark.java, htroot/yacysearch_location.java, lib/apache-solr-solrj-3.1.0.jar, lib/apache-solr-solrj-3.1.0.txt, lib/commons-httpclient-3.1.jar, lib/commons-httpclient-LICENSE.txt, lib/slf4j-LICENSE.txt, lib/slf4j-api-1.5.5.jar, lib/slf4j-jdk14-1.5.5.jar, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/SearchAccumulator.java, source/net/yacy/cora/services/federated/SearchHub.java, source/net/yacy/cora/services/federated/SearchResult.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/cora/services/federated/solr/SolrHTTPClient.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Thu Apr 14 00:04:23 CEST 2011 by orbiter | enhanced speed for OrderedScoreMap inc method and size comparisment in concurrent environments git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7653 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/DidYouMean.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/net/yacy/cora/storage/ClusteredScoreMap.java, source/net/yacy/cora/storage/ConcurrentScoreMap.java, source/net/yacy/cora/storage/OrderedScoreMap.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/WordCache.java |
Tue Apr 12 07:02:36 CEST 2011 by orbiter | more UTF8 getBytes() performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7649 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml, htroot/yacysearch_location.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/MessageBoard.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/UserDB.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java |
Fri Apr 08 23:15:10 CEST 2011 by apfelmaennchen | refactoring for ymarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7648 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkDate.java, source/de/anomic/data/ymark/YMarkHTMLImporter.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkUtil.java, source/de/anomic/data/ymark/YMarkWordCountComparator.java, source/de/anomic/data/ymark/YMarkXBELImporter.java |
Mon Apr 04 14:20:20 CEST 2011 by orbiter | fix for bug http://bugs.yacy.net/view.php?id=10 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7642 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/Latency.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/UserDB.java, source/de/anomic/http/client/Cache.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java |
Mon Apr 04 01:39:45 CEST 2011 by orbiter | added a handling of appearances of yacy bot entries in robots.txt if this entry addresses the yacy peer (directly or indirectly) and it grants a crawl-delay of 0. Then all forced pause mechanisms in YaCy are switched off and the domain is crawled at full speed. crawl delay values can be assigned to either - all yacy peers using the user-agent yacybot - a specific peer with peer name <peer-name>.yacy or - a specific peer with peer hash <peer-hash>.yacyh git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7639 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.java, htroot/api/util/getpageinfo_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/yacy/yacySeedDB.java |
Sat Apr 02 01:32:40 CEST 2011 by low012 | *) added new user right: extended search right (allows to define users who can query more results than anonymous users) *) cleaned up code a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7635 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/yacysearch.java, source/de/anomic/data/UserDB.java, source/de/anomic/search/Switchboard.java, source/net/yacy/ai/greedy/Asset.java |
Thu Mar 31 01:26:36 CEST 2011 by orbiter | the location search shows now not re-evaluated locations but only such locations that are attached as metadata to web pages - added parser for in-text appearing geo-locations - added geo-locations to rss search result - added evaluation of metadata-attached geo-locations in yacysearch_location to show search results within a map git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7631 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.rss, htroot/yacysearch_location.html, htroot/yacysearch_location.java, htroot/yacysearchitem.java, htroot/yacysearchitem.xml, source/de/anomic/search/ResultEntry.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/io/CharBuffer.java |
Wed Mar 30 02:49:47 CEST 2011 by orbiter | - added geo information parsing to html parser - extended metadata information in index with geolocalisation - added display of location in yacydoc and ViewFile git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7629 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/apicall.sh, htroot/IndexCreateWWWLocalQueue_p.html, htroot/ViewFile.html, htroot/ViewFile.java, htroot/api/yacydoc.html, htroot/api/yacydoc.java, htroot/api/yacydoc.xml, htroot/yacysearch_location.html, htroot/yacysearch_location.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/search/Segment.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/io/CharBuffer.java, source/net/yacy/yacy.java |
Tue Mar 22 11:35:26 CET 2011 by orbiter | enhancements to web cache and less strict caching rules git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7620 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/CacheResource_p.java, htroot/api/util/getpageinfo_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/OSMTile.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/repository/LoaderDispatcher.java |
Mon Mar 21 14:05:51 CET 2011 by orbiter | enhancement during search process: fast fail of search in case that all index feeder have terminated. This change should affect filtering and navigators and should cause that search navigation gets faster git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7614 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/kelondro/index/Row.java |
Mon Mar 21 08:50:34 CET 2011 by orbiter | added more configuration options for search: - removed configuration button for 'search only for admin' from index.html and added this to ConfigPortal - added configuration of link verification options (iffresh, cacheonly, nocache, ifexist) to ConfigPortal - added configuration of navigation options to ConfigPortal - added an option to switch off automatic index cleaning in case that a link verification method fails git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7613 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/index.html, htroot/index.java, htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java |
Sun Mar 20 00:52:09 CET 2011 by orbiter | - fixed missing save operation for peer name change - fixed import of mediawiki dump files - added script to add mediawiki dump files git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7609 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/importmediawiki.sh, htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigBasic.java, htroot/IndexImportWikimedia_p.html, htroot/IndexImportWikimedia_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/ViewProfile.java, htroot/Wiki.java, htroot/mediawiki_p.java, source/de/anomic/data/wiki/AbstractWikiParser.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/data/wiki/WikiParser.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/document/importer/MediawikiImporter.java |
Tue Mar 15 02:03:35 CET 2011 by low012 | *) simplified code\n*) fixed potential NumberFormatExceptions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7600 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessPicture_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigPortal.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CrawlResults.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexShare_p.java, htroot/Load_RSS_p.java, htroot/MessageSend_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/RemoteCrawl_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Table_API_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/tags/getTag.java, htroot/index.java, htroot/opensearchdescription.java, htroot/sharedBlacklist_p.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacySeed.java |
Tue Mar 15 00:33:36 CET 2011 by Florian Richter | * add copyright info * implement basic authentication * update jetty to 7.3.0 Changed Files: .classpath, build.xml, lib/jetty-continuation-7.3.0.v20110203.jar, lib/jetty-http-7.3.0.v20110203.jar, lib/jetty-io-7.3.0.v20110203.jar, lib/jetty-security-7.3.0.v20110203.jar, lib/jetty-server-7.3.0.v20110203.jar, lib/jetty-util-7.3.0.v20110203.jar, source/net/yacy/http/ContentModHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/SSIHandler.java, source/net/yacy/http/TemplateHandler.java, source/net/yacy/http/YaCyLoginService.java |
Sun Mar 13 02:41:44 CET 2011 by orbiter | redesigned the ScoreMap classes: - new concurrent score map using atom operation from java concurrency classes - redesigned difference beween StaticScore and Dynamic Score into ScoreMap and ReversibleScoreMap allowed that many classes can now use simple ScoreMap Objects which can be used better in concurrent environments using the ConcurrentScoreMap - switched from DynamicScore to ConcurrentScoreMap usage wherever possible git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7586 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Supporter.java, htroot/Surftips.java, htroot/yacy/search.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/dht/PeerSelection.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/storage/ClusteredScoreMap.java, source/net/yacy/cora/storage/ConcurrentScoreMap.java, source/net/yacy/cora/storage/IntScore.java, source/net/yacy/cora/storage/OrderedScoreMap.java, source/net/yacy/cora/storage/ReversibleScoreMap.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/document/WordCache.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/yacy.java |
Fri Mar 11 01:59:27 CET 2011 by orbiter | avoid string conversion git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7584 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/yacydoc.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/search/Segment.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/index/Row.java |
Fri Mar 11 00:25:07 CET 2011 by orbiter | - replaced more direct string-based UTF-8 conversions by predefined UTF-8 conversion - changed menu structure slightly git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7583 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigPortal.html, htroot/IndexControlCleaner_p.html, htroot/MessageSend_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/Wiki.java, htroot/api/ymarks/import_ymark.java, htroot/env/templates/header.template, htroot/sharedBlacklist_p.java, htroot/yacy/message.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/http/server/ChunkedOutputStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/FileUtils.java |
Thu Mar 10 13:35:32 CET 2011 by orbiter | moved getBytes() to UTF8.getBytes() to use a default String encoding git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7580 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, htroot/Supporter.java, htroot/Surftips.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segments.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/dbtest.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/ObjectBuffer.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/word/WordReference.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/CharBuffer.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/StringOrder.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/repository/LoaderDispatcher.java |
Wed Mar 09 17:32:34 CET 2011 by orbiter | more memory protection: auto-flush of caches in case of memory shortage git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7575 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/document/WordCache.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/table/Table.java |
Wed Mar 09 13:50:39 CET 2011 by orbiter | - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 1) if the file is changed for a re-crawl this is not reflected in the steering because it would take the previously uploaded crawl start file 2) browsers do not submit the full path of the selected file even if this path is shown in the input field because of security reasons. There is no work-around or hack to make the submission of the full path possible - fixed deletion of crawl start point urls in crawl stack and balancer double-check - fixed a problem with steering self-call (no resolving of localhost) - added more logging for the crawler to supervise why crawl urls are not taken by the loader - added a javascript onload-function to select domain restriction in all cases where a crawl is started from a file or from a url - fixed the restrict-to-domain pattern computation, added a 'www.'-prefix and added this functionality also to a crawl start from file git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7574 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartExpert_p.html, htroot/Crawler_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/data/WorkTables.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/visualization/RasterPlotter.java |
Mon Jan 17 12:06:17 CET 2011 by Florian Richter | * jetty Changed Files: .classpath, build.xml, lib/jetty-continuation-7.0.2.v20100331.jar, lib/jetty-http-7.0.2.v20100331.jar, lib/jetty-io-7.0.2.v20100331.jar, lib/jetty-server-7.0.2.v20100331.jar, lib/jetty-util-7.0.2.v20100331.jar, source/net/yacy/http/AuthenticationHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/TemplateHandler.java |
Wed Mar 09 10:29:05 CET 2011 by orbiter | always try to guess the size of a StringBuilder to prevent too many memory re-allocations git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7572 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/util/getpageinfo_p.java, htroot/yacysearchitem.java, source/de/anomic/data/ListManager.java, source/de/anomic/data/URLLicense.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/cora/document/JSONArray.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/Punycode.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/document/Document.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/logging/MiniLogFormatter.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MapTools.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java |
Tue Mar 08 02:51:51 CET 2011 by low012 | *) set SVN properties *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7567 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/WorkTables.java, source/de/anomic/net/UPnP.java, source/de/anomic/net/whois.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/ContentDomain.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/tools/ListDirs.java, source/de/anomic/tools/PKCS12Tool.java, source/de/anomic/tools/SignatureOutputStream.java, source/de/anomic/tools/bitfield.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/loaderCore.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/tarTools.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyChannel.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyUpdateLocation.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/example/Hanoi.java, source/net/yacy/ai/example/SchwarzerPeter.java, source/net/yacy/ai/example/testorder.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/AbstractModel.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Asset.java, source/net/yacy/ai/greedy/Attempts.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Challenge.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/ContextFactory.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/ai/greedy/Finding.java, source/net/yacy/ai/greedy/Goal.java, source/net/yacy/ai/greedy/Model.java, source/net/yacy/ai/greedy/Role.java, source/net/yacy/ai/greedy/Unirole.java, source/net/yacy/cora/date/AbstractFormatter.java, source/net/yacy/cora/date/DateFormatter.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/date/ISO8601Formatter.java, source/net/yacy/cora/document/ASCIIComparator.java, source/net/yacy/cora/document/Channel.java, source/net/yacy/cora/document/Channels.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/JSONArray.java, source/net/yacy/cora/document/JSONException.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/document/JSONTokener.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/cora/protocol/ByteArrayBody.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/GzipCompressingEntity.java, source/net/yacy/cora/protocol/http/GzipDecompressingEntity.java, source/net/yacy/cora/protocol/http/GzipRequestInterceptor.java, source/net/yacy/cora/protocol/http/GzipResponseInterceptor.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/cora/protocol/http/LinkExtractor.java, source/net/yacy/cora/protocol/http/ProxySettings.java, source/net/yacy/cora/services/SearchAccumulator.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ComparableARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/DynamicScore.java, source/net/yacy/cora/storage/HashARC.java, source/net/yacy/cora/storage/IntScore.java, source/net/yacy/cora/storage/OutOfLimitsException.java, source/net/yacy/cora/storage/ScoreCluster.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/cora/storage/StaticScore.java, source/net/yacy/cora/storage/WeakPriorityBlockingQueue.java, source/net/yacy/dbtest.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/gui/InfoPage.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/gui/framework/Application.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/gui/framework/Layout.java, source/net/yacy/gui/framework/Operation.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Gap.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/ObjectBuffer.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReference.java, source/net/yacy/kelondro/data/image/ImageReferenceFactory.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceVars.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceFactory.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/data/word/WordReference.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSpaceExceededException.java, source/net/yacy/kelondro/io/AbstractReader.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/ByteCount.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/CachedRecords.java, source/net/yacy/kelondro/io/Reader.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/io/Writer.java, source/net/yacy/kelondro/order/AbstractOrder.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/CloneableIterator.java, source/net/yacy/kelondro/order/CloneableMapIterator.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MicroDate.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/Order.java, source/net/yacy/kelondro/order/RotateIterator.java, source/net/yacy/kelondro/order/StringOrder.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/BufferedIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/IndexReader.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceContainerOrder.java, source/net/yacy/kelondro/rwi/ReferenceFactory.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/CompressedHashMap.java, source/net/yacy/kelondro/util/ConsoleInterface.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/Formatter.java, source/net/yacy/kelondro/util/ISO639.java, source/net/yacy/kelondro/util/LookAheadIterator.java, source/net/yacy/kelondro/util/MapTools.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/NamePrefixThreadFactory.java, source/net/yacy/kelondro/util/ObjectSpace.java, source/net/yacy/kelondro/util/ReverseMapIterator.java, source/net/yacy/kelondro/util/XMLTables.java, source/net/yacy/kelondro/util/kelondroException.java, source/net/yacy/kelondro/workflow/AbstractBlockingThread.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/kelondro/workflow/BlockingThread.java, source/net/yacy/kelondro/workflow/BusyThread.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/kelondro/workflow/WorkflowJob.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/kelondro/workflow/WorkflowThread.java, source/net/yacy/migration.java, source/net/yacy/repository/BlacklistFile.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, source/net/yacy/upnp/DiscoveryListener.java, source/net/yacy/upnp/HttpResponse.java, source/net/yacy/upnp/JXPathParser.java, source/net/yacy/upnp/ServiceEventMessageParser.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/devices/UPNPDevice.java, source/net/yacy/upnp/devices/UPNPRootDevice.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/ActionMessageResponseParser.java, source/net/yacy/upnp/messages/ActionResponse.java, source/net/yacy/upnp/messages/StateVariableMessage.java, source/net/yacy/upnp/messages/UPNPResponseException.java, source/net/yacy/upnp/services/ISO8601Date.java, source/net/yacy/upnp/services/ServiceAction.java, source/net/yacy/upnp/services/UPNPService.java, source/net/yacy/visualization/AnimationGIF.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/FontGenerator5Pixel.java, source/net/yacy/visualization/FontGenerator8Pixel.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/HexGridPlotter.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java, source/org/apache/tools/tar/TarEntry.java, source/org/apache/tools/tar/TarInputStream.java, source/org/apache/tools/tar/TarOutputStream.java |
Mon Mar 07 23:59:19 CET 2011 by low012 | *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7563 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/ConsoleOutHandler.java, source/net/yacy/kelondro/logging/GuiHandler.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/logging/MiniLogFormatter.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 07 21:42:19 CET 2011 by orbiter | reverted SVN 7557 because these classes are called using reflection. The class declaration is in the log configuration. Without these classes you get errors during runtime and a non-formatted log output, i.e.: STARTUP: Trying to load logging configuration from file /Data/workspace1/yacy/DATA/LOG/yacy.logging Can't load log handler "net.yacy.kelondro.logging.ConsoleOutErrHandler" java.lang.ClassNotFoundException: net.yacy.kelondro.logging.ConsoleOutErrHandler java.lang.ClassNotFoundException: net.yacy.kelondro.logging.ConsoleOutErrHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.util.logging.LogManager$3.run(LogManager.java:359) at java.security.AccessController.doPrivileged(Native Method) at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:346) at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:898) at java.util.logging.LogManager.access$900(LogManager.java:130) at java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:979) at java.util.logging.Logger.log(Logger.java:454) at java.util.logging.Logger.doLog(Logger.java:480) at java.util.logging.Logger.log(Logger.java:503) at net.yacy.kelondro.logging.Log$logRunner.run(Log.java:332) Can't load log handler "net.yacy.kelondro.logging.LogalizerHandler" java.lang.ClassNotFoundException: net.yacy.kelondro.logging.LogalizerHandler java.lang.ClassNotFoundException: net.yacy.kelondro.logging.LogalizerHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.util.logging.LogManager$3.run(LogManager.java:359) at java.security.AccessController.doPrivileged(Native Method) at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:346) at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:898) at java.util.logging.LogManager.access$900(LogManager.java:130) at java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:979) at java.util.logging.Logger.log(Logger.java:454) at java.util.logging.Logger.doLog(Logger.java:480) at java.util.logging.Logger.log(Logger.java:503) at net.yacy.kelondro.logging.Log$logRunner.run(Log.java:332) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7559 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/ConsoleOutHandler.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/logging/MiniLogFormatter.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java |
Mon Mar 07 21:36:40 CET 2011 by orbiter | replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7558 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigNetwork_p.java, htroot/CrawlResults.java, htroot/CrawlStartScanner_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/Load_RSS_p.java, htroot/Messages_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Table_API_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/ViewFile.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/queues_p.java, htroot/api/table_p.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/mediawiki_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/message.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/MessageBoard.java, source/de/anomic/data/UserDB.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java, source/de/anomic/data/list/XMLBlacklistImporter.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/gzip.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/cora/storage/ScoreCluster.java, source/net/yacy/dbtest.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/CachedRecords.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/CompressedHashMap.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java, test/de/anomic/crawler/robotsParserTest.java |
Mon Mar 07 02:45:11 CET 2011 by orbiter | more patches for a better out-of-memory management git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7555 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/kelondro/blob/Gap.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/visualization/RasterPlotter.java |
Fri Mar 04 14:44:00 CET 2011 by orbiter | introduction of a new remote search load control: the remote search has taken 10 results per peer with a time-out of 3 seconds so far. The attributes of number of results per peer and time-out time can now be configured. This has two aspects: the user who searches may want to increase these values to get more results and more load on the remote side and the user of the server which is accessed for this search may want to restrict the load. Both sides can now be configured. The server-site maximum load parameters are defined by a network definition and the client-side search request load can be defined by each user individually but when the remote search is done the requested service is limited to the network definition. You can find now in the network definition file: network.unit.remotesearch.maxcount and network.unit.remotesearch.maxtime and in the yacy.conf file: remotesearch.maxcount and remotesearch.maxtime There is currently no web interface to define the client-side remote search attributes, please set them manually git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7548 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, defaults/yacy.network.freeworld.unit, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java |
Tue Mar 01 10:03:33 CET 2011 by orbiter | - fixed a memory leak in the httpc.post method (no finish) - patched some more memory-saving relevant code - some more minor bug fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7541 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/kelondro/blob/HeapReader.java |
Mon Feb 28 01:22:05 CET 2011 by mikeworks | de.lng: Updated German translation and removed old unused strings, e.g. 8080 -> 8090 and Search Portal translations Bookmarks.html, Ranking_p.html, base.css: Fixed XHTML errors to make pages compatible again - switched div -> span inside dt and replaced css definition of id (unique) with class (class of elements) header.template: Fixed link to berliOS changelog by replacing & -> & and adding translation for German page by refering to German berliOS UI ;-] git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7539 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.html, htroot/Ranking_p.html, htroot/env/base.css, htroot/env/templates/header.template, locales/de.lng |
Sat Feb 26 02:24:49 CET 2011 by orbiter | some more performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7533 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/imagetest.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/OSMTile.java, source/net/yacy/kelondro/blob/Gap.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java |
Fri Feb 25 22:11:53 CET 2011 by orbiter | continuing the fight against deadlocks during time formatting: better caching. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7531 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ContentIntegrationPHPBB3_p.java, htroot/IndexControlURLs_p.java, htroot/api/bookmarks/posts/get.java, htroot/cytag.java, htroot/yacy/query.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/WorkTables.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/date/AbstractFormatter.java, source/net/yacy/cora/date/DateFormatter.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/date/ISO8601Formatter.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/table/SplitTable.java |
Fri Feb 25 13:41:27 CET 2011 by orbiter | performance hack: replaced generic row.getColBytes() call with row.getPrimaryKeyBytes() where the column is 0 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7529 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/util/FileUtils.java |
Fri Feb 25 13:23:00 CET 2011 by orbiter | - enhanced date formatter cache - added more instances of formatter objects to different classes to make them independent in case of lockings that may applay during synchronization of the date formatter object (date formatting is not thread-safe and must be synchronized therefore) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7528 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java |
Fri Feb 25 01:56:31 CET 2011 by orbiter | Enhanced Base64Order transformation: less overhead (transformation between StringBuilder and byte[]) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7523 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/yacyNewsDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/MicroDate.java |
Wed Feb 23 01:32:16 CET 2011 by orbiter | - removed locks from WordReference - refactoring of HeapReader/Writer git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7514 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceVars.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/Reference.java |
Sat Feb 19 02:56:12 CET 2011 by orbiter | more animations in graphics. See network and access picture. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7498 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessGrid_p.html, htroot/AccessPicture_p.java, htroot/imagetest.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/HexGridPlotter.java, source/net/yacy/visualization/RasterPlotter.java |
Fri Feb 18 02:46:07 CET 2011 by low012 | *) deleted LogStatistics since the page did not work anymore and it seemed to be obsolete, tell me if you miss it and I will add it again *) a few minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7494 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexImportWikimedia_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.java, source/de/anomic/crawler/Balancer.java |
Mon Feb 14 23:57:43 CET 2011 by orbiter | added accept-charset="UTF-8" to all forms this applies patches from http://forum.yacy-websuche.de/viewtopic.php?p=20891#p20891 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7482 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/BlacklistCleaner_p.html, htroot/BlacklistImpExp_p.html, htroot/BlacklistTest_p.html, htroot/Blacklist_p.html, htroot/Blog.html, htroot/Bookmarks.html, htroot/Collage.html, htroot/ConfigAccounts_p.html, htroot/ConfigAppearance_p.html, htroot/ConfigBasic.html, htroot/ConfigHTCache_p.html, htroot/ConfigLanguage_p.html, htroot/ConfigPortal.html, htroot/ConfigProperties_p.html, htroot/ConfigUpdate_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/CookieMonitorIncoming_p.html, htroot/CookieMonitorOutgoing_p.html, htroot/CrawlProfileEditor_p.html, htroot/CrawlResults.html, htroot/Crawler_p.html, htroot/DictionaryLoader_p.html, htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/IndexCreateDomainCrawl_p.html, htroot/IndexCreateParserErrors_p.html, htroot/IndexCreateWWWGlobalQueue_p.html, htroot/IndexCreateWWWLocalQueue_p.html, htroot/IndexCreateWWWRemoteQueue_p.html, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportWikimedia_p.html, htroot/IndexShare_p.html, htroot/Load_RSS_p.html, htroot/Messages_p.html, htroot/Network.html, htroot/News.html, htroot/PerformanceQueues_p.html, htroot/Performance_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/Status.html, htroot/Surftips.html, htroot/Table_API_p.html, htroot/Table_YMark_p.html, htroot/Tables_p.html, htroot/Threaddump_p.html, htroot/User.html, htroot/ViewLog_p.html, htroot/WatchWebStructure_p.html, htroot/compare_yacy.html, htroot/index.html, htroot/sharedBlacklist_p.html, htroot/www/welcome.html |
Sun Feb 13 18:37:28 CET 2011 by orbiter | introduction of dht-burst modes: this can expand the number of target peers in some cases where a better heuristic is needed. The problematic cases are either when a muti-word search is made (still a hard case for our term-oriented DHT) or when a network operator wants that all robinson peers are asked. We therefore introduced two new network steering values that switch on more peers during the peer selection. Because the number of peers can now be very large, the number of maximum httpc connections was also increased. Please see new coments in yacy.network.freeworld.unit for details of the new DHT selection methods. The number of maximum peers is now not fixed to a specific number but may increase with - the partition exponent - the number of redundant peers - the robinson burst percentage - the multiword burst percentage The maximum can then be the number of senior peers (all visible peers). git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7479 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.freeworld.unit, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/yacy/dht/PartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/WordTokenizer.java |
Sat Feb 12 01:01:40 CET 2011 by orbiter | - fixed document number limitation for crawls that restrict the number of documents per domain - some restructuring of the document counting and logging structures was necessary - better abstraction of CrawlProfiles - added deletion of logs to the index deletion option (if the index is deleted using the servlets) which is necessary to reset the domain counters for the page limitation - more refactoring to get the LibraryProvider more clean - some refactoring of the Condenser class git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7478 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/WatchWebStructure_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/search.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/WordCache.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/repository/LoaderDispatcher.java |
Tue Feb 08 22:46:24 CET 2011 by sixcooler | update to httpclient-4.1 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7473 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/httpclient-4.1.License, lib/httpclient-4.1.jar, lib/httpcore-4.1.License, lib/httpcore-4.1.jar, lib/httpmime-4.1.License, lib/httpmime-4.1.jar |
Wed Feb 02 16:54:13 CET 2011 by orbiter | added flags and administration options to show advanced search and to show search result attributes (for each search result) Administration can be done at ConfigPortal.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7466 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/env/templates/header.template, htroot/env/templates/submenuPortalIntegration.template, htroot/index.html, htroot/index.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/server/serverObjects.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java |
Wed Feb 02 01:06:29 CET 2011 by orbiter | replaced more appearance of double values by float values git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7461 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/ConfigUpdate_p.java, htroot/yacysearch.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyBuildProperties.java, source/net/yacy/cora/document/JSONArray.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/Location.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/yacy.java |
Wed Feb 02 00:49:11 CET 2011 by orbiter | moved all Double values to Float values because of http://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/ YaCy does not really need double-precision floating point computation anywhere, so this should not affect any feature git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7460 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigUpdate_p.java, htroot/Network.java, htroot/PerformanceQueues_p.java, htroot/osm.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/util/Formatter.java |
Fri Jan 28 11:54:13 CET 2011 by orbiter | changed the default port from 8080 to 8090 see also: http://forum.yacy-websuche.de/viewtopic.php?p=21683#p21683 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7454 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy-Search.bat, addon/YaCy-Search.html, addon/yacy_chatscript.pl, bin/graphicstest.sh, defaults/yacy.init, htroot/ConfigBasic.java, htroot/ConfigLiveSearch.java, htroot/ConfigPortal.java, htroot/ConfigSearchBox.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartScanner_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Network.html, htroot/ProxyIndexingMonitor_p.html, htroot/QuickCrawlLink_p.java, htroot/Settings_ProxyAccess.inc, htroot/Settings_p.java, htroot/Status.html, htroot/Status.java, htroot/Table_API_p.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/processing/domaingraph/applet/domaingraph.pde, htroot/processing/domaingraph/domaingraph.pde, htroot/rssTerminal.html, htroot/sharedBlacklist_p.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/hello.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearch_location.java, readme.txt, source/de/anomic/data/WorkTables.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/yacyChannel.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/gui/InfoPage.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/yacy.java, startYACY.command, startYACY.sh, startYACY_debug.bat |
Fri Jan 28 11:12:17 CET 2011 by orbiter | - removed 'peerName' property from the yacy settings file because this information is stored in the yacy seed file - the own seed file gets the lead for storage of the peer name - exchanged default peer name generation method with one that does not use the local ip - default peer names are now strings starting with '_anon' - added another switch to suppress forwarding to ConfigBasic if the name was already changed - replaced all usages of the yacy.conf peerName with access to the local seed - changes to the peer name are now applied directly and not after the next peer ping git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7453 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/Settings_p.java, htroot/Status.java, htroot/www/welcome.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacySeed.java |
Thu Jan 27 00:21:33 CET 2011 by orbiter | extended seed list uploader: do not only upload all active peers but also some more peers that are passive but had been active in the last 24 hours git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7449 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/ChunkedInputStream.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacySeedUploader.java |
Sat Jan 22 10:46:00 CET 2011 by orbiter | modified auto-delete of search failure-words: - words are now not deleted from the search index automatically if index receive is switched off - a flag in the network definition defines if this feature is switched on at all - the search filter for not-found word references is switched off for server-side remote searches git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7441 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.webportal.unit, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java |
Mon Jan 17 12:37:30 CET 2011 by f1ori | * adminAccountForLocalhost only for localhost * yacy crawls local domains also, if no password is set (the interface is already protected) * it's not required anymore, to set a password in intranet mode git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7436 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAccounts_p.java, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/repository/LoaderDispatcher.java |
Wed Jan 12 01:00:14 CET 2011 by orbiter | - cleaned up yacy news - removed unused methods - avoid news generation in case that the peer runs in robinson mode git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7431 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blog.java, htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/Network.html, htroot/Network.java, htroot/Network.xml, htroot/Supporter.java, htroot/Surftips.java, htroot/Wiki.java, htroot/api/bookmarks/posts/add_p.java, htroot/yacysearch.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacyVersion.java |
Tue Jan 11 23:58:14 CET 2011 by orbiter | - more control in access tracker: count number of returned search results (not only info how much is in the index) - extended query params for this - enhanced cora git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7430 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java, htroot/AccessTracker_p.xml, htroot/yacy/search.java, htroot/yacysearch.rss, htroot/yacysearch_location.java, htroot/yacysearchitem.java, source/de/anomic/search/QueryParams.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/repository/LoaderDispatcher.java |
Mon Jan 10 21:24:52 CET 2011 by low012 | *) Minor changes, mainly cleaning up a little bit, no functional changes. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7428 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/document/parser/images/icoParser.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/DemoApplet.java, source/net/yacy/visualization/FontGenerator8Pixel.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java |
Mon Jan 03 21:52:54 CET 2011 by orbiter | - cora package has now no dependencies to other yacy packages and becomes a 'base' package (refactoring) - cleaned up (removed special code and documentation for 27c3) - added remote search functions to be used within cora git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7420 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CrawlStartScanner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Network.java, htroot/News.java, htroot/RemoteCrawl_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/latency_p.java, htroot/api/timeline.java, htroot/api/ymarks/get_treeview.java, htroot/cytag.java, htroot/yacy/query.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.html, htroot/yacysearch_location.java, htroot/yacysearchitem.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/http/server/AlternativeDomainNames.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/date/AbstractFormatter.java, source/net/yacy/cora/date/DateFormatter.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/date/ISO8601Formatter.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/protocol/http/LinkExtractor.java, source/net/yacy/cora/protocol/http/ProxySettings.java, source/net/yacy/cora/services/Search.java, source/net/yacy/cora/storage/IntScore.java, source/net/yacy/cora/storage/OutOfLimitsException.java, source/net/yacy/cora/storage/ScoreCluster.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/yacy.java |
Mon Dec 27 18:07:21 CET 2010 by low012 | *) cleaning up the code a little bit *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7396 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewFile.java, source/de/anomic/crawler/ResultImages.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/Dao.java, source/net/yacy/document/content/dao/DatabaseConnection.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/language/LanguageFilenameFilter.java, source/net/yacy/document/language/LanguageStatistics.java, source/net/yacy/document/language/LanguageStatisticsHolder.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java |
Mon Dec 27 16:37:11 CET 2010 by low012 | *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7394 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/zipParser.java |
Fri Dec 17 18:30:09 CET 2010 by orbiter | - fixed numerous bugs - better document names - fixed problem with ftp crawling - added automatic removal of search results from services that are not online according to the latest network scan: this does not delete the index but just does not show them. after the next network scan when the server is available again, the results are again showed. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7385 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/search/RankingProcess.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/parser/html/ContentScraper.java |
Fri Dec 17 00:37:21 CET 2010 by orbiter | - a collection of bug fixes and some redesign of the Scanner class - fixed smb crawling - added smbget to download script generation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7381 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/js/yacyinteractive.js, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segments.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java |
Wed Dec 15 20:20:00 CET 2010 by f1ori | * fix system update if urls are in blacklist (for example for very general blacklists like *.de) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7375 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/DictionaryLoader_p.java, htroot/Load_RSS_p.java, htroot/ViewFile.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/import_ymark.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/OSMTile.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/repository/LoaderDispatcher.java |
Wed Dec 15 01:03:19 CET 2010 by orbiter | - added CamelCase parser to MultiProtocolURI: generate better to-be-indexed words from urls - integrated new parser into loader processes: enrich document parser - fixed a concurrent modification exception in kelondro iterator - hand-over of document size from crawler to indexer git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7374 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/kelondro/util/ReverseMapIterator.java |
Sat Dec 11 01:31:57 CET 2010 by orbiter | enhanced crawler: - added a new queue 'noload' which can be filled with urls where it is already known that the content cannot be loaded. This may be because there is no parser available or the file is too big - the noload queue is emptied with the parser process which indexes the file names only - the 'start from file' functionality now also reads from ftp crawler git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7368 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/api/queues_p.java, htroot/rct_p.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/TextParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/repository/LoaderDispatcher.java |
Thu Dec 09 18:17:25 CET 2010 by orbiter | added a very fast ftp file list generator to site crawler: - when a site-crawl for ftp sites is now started, then a special directory-tree harvester gets the complete directory structure of a ftp server at once - the harvester runs concurrently and feeds into the normal crawl queue also in this: - fixed the 'start from file' crawl function - added a link detector for the html parser. The html parser can now also extract links that are not included in <a> tags. - this causes that a crawl start is now also possible from clear text link files git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7367 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/js/IndexCreate.js, source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/parser/html/ContentScraper.java |
Wed Dec 08 11:50:23 CET 2010 by orbiter | removed the display option from index.html, yacysearch.html and yacyinteractive.html instead, a setting at ConfigPortal.html can be made to define if the topmenu shall be shown at these pages or if there is no naviagtion at all. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7366 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/env/templates/header.template, htroot/index.html, htroot/index.java, htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacyinteractive.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/QueryParams.java |
Tue Dec 07 16:46:04 CET 2010 by orbiter | added a forwarder for the default page. The forwarder forwards a browser to a different page if the root file index.html is accessed. This can be done by setting the name of the forwarder page to the field "Default index.html Page (by forwarder)" in /ConfigPortal.html The purpose is to forward to /yacyinteractive.html for the 27C3 FTP search plattform git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7365 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/index.html, htroot/index.java, source/de/anomic/search/SwitchboardConstants.java |
Mon Dec 06 15:34:58 CET 2010 by orbiter | some redesign of the search-fail-response mechanism: when a search fails for a single url because the snippet cannot be generated, then the url reference is deleted from the index. This mechanism was redesign and enhanced. The process now also writes into the work tables into the table searchfl to prepare a re-indexing mechanism. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7364 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/WorkTables.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java |
Mon Dec 06 00:54:00 CET 2010 by orbiter | fixed several search result navigation bugs fixed bad behaviours during search result collection git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7362 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacy/search.java, htroot/yacyinteractive.html, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java |
Thu Dec 02 12:05:04 CET 2010 by orbiter | fixed bugs in parser and ftp client git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7360 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java |
Wed Dec 01 19:48:21 CET 2010 by orbiter | added image search to yacyinteractive.html this causes that the search result view switches from list format to image preview format when a search is restricted to png, gif or jpg documents git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7358 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchitem.json, source/de/anomic/search/QueryParams.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java |
Tue Nov 30 17:13:55 CET 2010 by orbiter | - added a catch-all parser for all documents that cannot be parsed: they will contributed with their document url for the search index only - enhanced the pdf and torrent parser: better documents titles - enhanced the ftp client: more time-out time - fixed bugs in json for search results - enhanced yacyinteractive.html: added a file type navigator and a download-script generator for search result files Please have a look at yacyinteractive.html: this will become the hacker-download tool for 27c3! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7355 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.html, htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacysearchtrailer.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/torrentParser.java |
Mon Nov 29 19:08:20 CET 2010 by orbiter | - added http://blekko.com as search heuristic (like scroogle). This was easy since they deliver their search results also as rss feed - renamed YaCys search result modifications keywords for RECENT, NEAR and language: to the blekko slashtag naming scheme. YaCy now supports the following blekko-like slash built-in slashtags: /date - for search results ordered by date (most recent up) /near - for search results where search words appear near to each other (closest up) /language/<lang> - for a sorting by language where the wanted language gets up. Example: /language/de git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7350 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigHeuristics_p.html, htroot/ConfigHeuristics_p.java, htroot/yacysearch.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/RSSReader.java |
Mon Nov 29 12:07:42 CET 2010 by orbiter | removed unused CR accumulation and distribution process this was never used and extended in the last years. The resulting YBR ranking criteria is still a good idea and will be used in the future. Possible generation methods for YBR ranking are: - "trust-rank" using the link structure as can be discovered in a single crawl (idea from FSCONS) - "block-rank" calculated from the local link structure - a distributed "block-rank" using the xml API to the link structure from other peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7349 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/Network.html, htroot/Network.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/yacy.java |
Sun Nov 28 03:57:31 CET 2010 by low012 | *) cleaning up the code a little bit *) program to interface, not implementation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7345 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/repository/Blacklist.java |
Sat Nov 27 01:54:59 CET 2010 by low012 | *) cleaning up the code a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7343 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessTracker_p.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigHTCache_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProperties_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CookieTest_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/Crawler_p.java, htroot/yacy/crawlReceipt.java, source/de/anomic/data/BookmarkDate.java |
Wed Nov 24 02:31:41 CET 2010 by low012 | *) cleaning up the code a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7336 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/config_p.java |
Sun Nov 21 04:39:53 CET 2010 by low012 | *) cleaner code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7331 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/BlacklistCleaner_p.java, htroot/api/bookmarks/posts/get.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/wiki/WikiCode.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/repository/Blacklist.java |
Sun Nov 21 02:29:32 CET 2010 by low012 | *) renamed classes according to standard Java coding conventions *) String.isEmpty() was introduced in Java 1.6, but we still use Java 1.5 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7330 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/Messages_p.java, htroot/User.java, htroot/Wiki.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/sharedBlacklist_p.java, htroot/yacy/message.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/BookmarksDB.java, source/de/anomic/data/Diff.java, source/de/anomic/data/ListManager.java, source/de/anomic/data/MessageBoard.java, source/de/anomic/data/Translator.java, source/de/anomic/data/UserDB.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/Switchboard.java, source/net/yacy/yacy.java |
Wed Nov 10 23:53:27 CET 2010 by apfelmaennchen | added a ymark table administration: /Table_YMark_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7316 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, htroot/Table_YMark_p.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksXBELImporter.java |
Tue Nov 09 18:01:31 CET 2010 by apfelmaennchen | more updates to ymarks - working xbel import/export - exported xbel includes yacy specific metadata but still validates against PUBLIC DTD git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7315 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_xbel.xml, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/info.html, htroot/api/ymarks/test_import.html, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksXBELImporter.java |
Sat Nov 06 21:26:13 CET 2010 by apfelmaennchen | ymarks - some refactoring - working xbel and html import (/api/ymarks/test_import.html) - working treeview (/api/ymarks/test_treeview.html) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7312 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_treeview.json, htroot/api/ymarks/get_treeview.xml, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_xbel.xml, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/import_ymark.xml, htroot/api/ymarks/test_import.html, htroot/api/ymarks/test_treeview.html, htroot/yacy/ui/css/jquery.treeview.css, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java, source/de/anomic/data/YMarksXBELImporter.java |
Commit | Description |
---|
Commit | Description |
---|---|
Sun Nov 27 17:57:33 CET 2011 by orbiter | YaCy 1.0 :) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8118 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.properties |
Sun Nov 27 17:54:48 CET 2011 by orbiter | removed unnecessary log entries git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8117 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/http/client/Cache.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/yacy.java |
Sun Nov 27 17:17:31 CET 2011 by orbiter | added a hint that the interactive search is only searching in the local index git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8116 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacyinteractive.html |
Sun Nov 27 17:01:30 CET 2011 by orbiter | added link to tutorial movies git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8115 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: readme.txt |
Sun Nov 27 16:58:52 CET 2011 by orbiter | added RAM and disk space hint git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8114 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: readme.txt |
Sun Nov 27 16:38:37 CET 2011 by orbiter | fixed page navigation counter git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8113 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacysearch.js, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchlatestinfo.java, htroot/yacysearchlatestinfo.json |
Sun Nov 27 15:00:16 CET 2011 by orbiter | remove more news (all older than one day) because they can be a performance problem if we have too many peers sending news git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8112 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/NewsPool.java |
Sun Nov 27 00:51:56 CET 2011 by orbiter | corrected the changelog link git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8111 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigUpdate_p.html |
Sun Nov 27 00:41:49 CET 2011 by orbiter | java 1.6 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8110 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: readme.txt |
Sun Nov 27 00:29:33 CET 2011 by orbiter | documentation update git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8109 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: readme.txt |
Sat Nov 26 12:31:22 CET 2011 by orbiter | added search operator help git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8107 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/base.css, htroot/index.html |
Fri Nov 25 19:08:42 CET 2011 by apfelmaennchen | small optical enhancements to ymarks treeview git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8106 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, htroot/env/yacy-ymarks.css |
Fri Nov 25 13:34:14 CET 2011 by orbiter | removed superfluous ranking tables (update will follow) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8102 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: |
Fri Nov 25 01:32:18 CET 2011 by apfelmaennchen | further update to YMarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8100 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/import_ymark.java, htroot/env/yacy-ymarks.css, htroot/jquery/js/jquery.multiselect.min.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkUtil.java |
Fri Nov 25 00:28:40 CET 2011 by orbiter | fixed link to development change log git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8099 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/header.template |
Thu Nov 24 23:45:31 CET 2011 by orbiter | fixed secondary remote search (the process that finds distributed join situations) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8098 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/SearchEvent.java |
Thu Nov 24 23:44:54 CET 2011 by sixcooler | Ant-Task for getting version from git tries to find svn-version or any tag - what ever comes first be careful using this with non-numerical tags! Changed Files: build.xml, libbuild/GitRevTask/GitRevTask.java, libbuild/GitRevTask/GitRevTask.properties, libbuild/org.eclipse.jgit-1.1.0.201109151100-r.jar, libbuild/org.eclipse.jgit-1.1.0.201109151100-r.license |
Thu Nov 24 21:07:52 CET 2011 by orbiter | new default ranking profile git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8097 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Ranking_p.java, source/net/yacy/search/ranking/RankingProfile.java |
Thu Nov 24 19:26:53 CET 2011 by sixcooler | test for pushing some files to ignore Changed Files: .gitignore |
Thu Nov 24 19:12:01 CET 2011 by sixcooler | Merge remote branch 'origin/master' Changed Files: htroot/yacysearch.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/search/ranking/ReferenceOrder.java |
Thu Nov 24 18:37:04 CET 2011 by orbiter | speed enhancements when parsing RWI rows (makes search slightly faster) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8096 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/search/ranking/ReferenceOrder.java |
Thu Nov 24 17:43:26 CET 2011 by orbiter | addon to http://bugs.yacy.net/view.php?id=72 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8095 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java |
Thu Nov 24 17:29:35 CET 2011 by orbiter | added a security check git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8094 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/Seed.java |
Thu Nov 24 17:05:09 CET 2011 by orbiter | fixed unresolved pattern and unwanted local/global switch when using votes on search results git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8093 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.java, htroot/yacysearch.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/query/SnippetProcess.java |
Thu Nov 24 15:57:09 CET 2011 by orbiter | creating a bookmark from the search results now works again .. with new YMarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8092 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.java, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/data/ymark/YMarkTables.java, source/net/yacy/search/query/QueryParams.java |
Thu Nov 24 15:51:25 CET 2011 by sixcooler | Ant-Task for getting version from git Changed Files: .gitignore, build.properties, build.xml, libbuild/GitRevTask/GitRevTask.java, libbuild/GitRevTask/GitRevTask.properties, libbuild/org.eclipse.jgit-1.1.0.201109151100-r.jar, libbuild/org.eclipse.jgit-1.1.0.201109151100-r.license |
Thu Nov 24 14:30:42 CET 2011 by orbiter | - removed lower page navigation (it never looks nice) - added visibility of metadata and parser in search results since that shows what YaCy can do in a nice way git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8091 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.java, htroot/yacysearch.html |
Thu Nov 24 14:07:03 CET 2011 by orbiter | - removed option to search for audio, video and applications. These things are still experimental and should not be shown to new users since this would cause them to argue that YaCy does not work. The functions are stil available, because: - added a configuration option in ConfigPortal to swtich the search media types on or off git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8090 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java |
Thu Nov 24 13:55:09 CET 2011 by orbiter | update to network graphics: slightly less crawling activity, slightly stronger color for query activity git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8089 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/graphics/NetworkGraph.java |
Thu Nov 24 11:29:20 CET 2011 by orbiter | fix for web structure if there is no web structure git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8088 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/WatchWebStructure_p.java |
Thu Nov 24 11:22:02 CET 2011 by orbiter | - better default values for caching and cache usage - set new caching and verification behavior according to use case automatically git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8087 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/ConfigBasic.java, htroot/ConfigPortal.java, htroot/WatchWebStructure_p.java, source/net/yacy/search/SwitchboardConstants.java |
Thu Nov 24 10:50:30 CET 2011 by orbiter | fix for language fix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8086 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataRow.java |
Thu Nov 24 02:30:12 CET 2011 by orbiter | bug fixes and speed enhancements for search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8085 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java, source/net/yacy/search/query/RWIProcess.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SnippetProcess.java |
Thu Nov 24 00:39:34 CET 2011 by orbiter | much better timing when search globally; less blocking; more results earlier! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8084 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, source/net/yacy/search/query/RWIProcess.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SnippetProcess.java |
Wed Nov 23 23:21:14 CET 2011 by orbiter | fixed cluster search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8083 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/query/SearchEvent.java |
Wed Nov 23 23:03:08 CET 2011 by orbiter | fix for bad language encoding git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8082 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataRow.java |
Wed Nov 23 22:44:56 CET 2011 by orbiter | argh.. fixed bad SVN 8080 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8081 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java |
Wed Nov 23 22:40:11 CET 2011 by orbiter | removed limitation to cluster peers if peer is asked remotely. This enables single-linked clusters which naturaly is there first if a new cluster is created git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8080 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java |
Wed Nov 23 22:38:57 CET 2011 by orbiter | added grafics for tutorial page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8079 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/grafics/flag_deutsch_28x17.gif, htroot/env/grafics/flag_english_28x17.gif |
Wed Nov 23 21:13:57 CET 2011 by orbiter | moved the Save button git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8078 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigNetwork_p.html |
Wed Nov 23 16:51:39 CET 2011 by orbiter | moved the help menu point (it is not any more related to search but to YaCy itself) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8077 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/header.template |
Wed Nov 23 16:45:05 CET 2011 by orbiter | removed Mac command file becuase the latest Mac OS does not support this. we will have a Mac Application instead git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8076 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: |
Wed Nov 23 16:43:49 CET 2011 by orbiter | replaced old help page with tutorial movies git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8075 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Help.html, htroot/env/templates/header.template, locales/de.lng |
Wed Nov 23 16:17:46 CET 2011 by orbiter | fix for compare search: use scroogle instead of bing and get a default search if configured search engine is not available git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8074 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/compare_yacy.java |
Wed Nov 23 16:07:10 CET 2011 by orbiter | removed google from compare search because google does not allow embedding of the search page in iframes (it just does not work any more) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8073 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/compare_yacy.java |
Tue Nov 22 16:17:24 CET 2011 by sixcooler | manual merge Changed Files: htroot/Status.html |
Tue Nov 22 13:12:25 CET 2011 by lotus | temporary remove compare search from tray git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8070 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/gui/Tray.java |
Tue Nov 22 13:04:24 CET 2011 by lotus | fix start on fresh windows. reason: -XX:ReservedCodeCacheSize=1024m may be too large (introduced in 7823) error message: could not create virtual machine there are more differences from the _debug.bat start script for the start parameters please check whether they are needed: -Xss256k -XX:MaxPermSize=256m -XX:+UseAdaptiveSizePolicy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8069 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: startYACY.bat |
Tue Nov 22 11:38:32 CET 2011 by orbiter | added api link to api page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8068 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_API_p.html |
Tue Nov 22 11:36:20 CET 2011 by orbiter | added api link to YMarks.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8067 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html |
Tue Nov 22 11:18:56 CET 2011 by orbiter | added api icon in table administration which appears automatically if a table was selected git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8066 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Tables_p.html |
Tue Nov 22 11:18:13 CET 2011 by orbiter | fixed xml output for table retrieval git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8065 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/table_p.java |
Tue Nov 22 01:22:09 CET 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=72 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8064 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java |
Tue Nov 22 01:03:20 CET 2011 by orbiter | add a bookmark automatically each time a new crawl is started git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8063 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/api/getpageinfo_p.java, htroot/compare_yacy.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/BookmarksDB.java |
Tue Nov 22 00:10:29 CET 2011 by orbiter | produce a bookmark entry from every crawl start. these bookmarks are always private. these bookmarks will be used to get a source reference for the search in case of intranet or portal searches. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8062 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/CrawlStartExpert_p.html, htroot/CrawlStartSite_p.html, htroot/Crawler_p.java, source/de/anomic/crawler/RobotsTxt.java |
Mon Nov 21 15:02:14 CET 2011 by orbiter | added a 'fork me' banner to motivate people to get hands on the code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8061 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Status.html, htroot/env/grafics/forkme.png |
Sun Nov 20 16:20:57 CET 2011 by cominch | Minor correction to prevent useless comma at beginning of string, created from list git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8059 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/ListManager.java |
Thu Nov 17 14:40:22 CET 2011 by orbiter | code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8055 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/SettingsAck_p.java, source/de/anomic/http/server/HTTPDProxyHandler.java |
Thu Nov 17 02:05:45 CET 2011 by orbiter | local/global fixes in search, better abstraction git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8054 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java |
Thu Nov 17 01:19:14 CET 2011 by orbiter | enhancements for extended search options git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8053 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.html, htroot/index.java |
Thu Nov 17 00:10:53 CET 2011 by apfelmaennchen | YMarks: - working direct importer for YaCy Crawl Starts - working direct import for old bookmarks.db git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8052 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/import_ymark.java, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, source/de/anomic/data/ymark/YMarkEntry.java, source/de/anomic/data/ymark/YMarkTables.java |
Wed Nov 16 23:57:15 CET 2011 by orbiter | fix for global search attribute when selecting extended search options git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8051 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.html |
Wed Nov 16 16:36:30 CET 2011 by orbiter | fix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8050 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/HTTPLoader.java |
Wed Nov 16 16:33:55 CET 2011 by orbiter | added also a default accept field to robots.txt downloads git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8049 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java |
Wed Nov 16 16:27:43 CET 2011 by orbiter | added a default accept header in http requests since some http fraud detection functions check that this header field exist see also: http://bad-behavior.ioerror.us/ in source file browser.inc.php git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8048 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, defaults/yacy.network.allip.unit, source/de/anomic/crawler/retrieval/HTTPLoader.java |
Wed Nov 16 15:09:50 CET 2011 by orbiter | more logging git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8047 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java, source/net/yacy/search/Switchboard.java |
Wed Nov 16 14:56:31 CET 2011 by orbiter | more logging for debugging of robots.txt git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8046 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java |
Wed Nov 16 14:12:46 CET 2011 by orbiter | fix for robot parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8045 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxtParser.java |
Wed Nov 16 14:06:46 CET 2011 by orbiter | fix for robot parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8044 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java |
Wed Nov 16 02:03:49 CET 2011 by orbiter | - enhanced logging in robots.txt parser for remote debugging - robots.txt is now more robust against database operations git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8043 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/getpageinfo_p.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/RobotsTxtEntry.java, source/net/yacy/search/Switchboard.java |
Wed Nov 16 01:21:56 CET 2011 by orbiter | moved ynetSearch to get all files out of htroot/api/util/ git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8042 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, htroot/api/ynetSearch.java, htroot/api/ynetSearch.json, htroot/api/ynetSearch.xml, htroot/yacy/ui/sidebar/sidebar_1.html |
Tue Nov 15 13:22:19 CET 2011 by cominch | getpageinfo: Checks if there is a OAI repository behind the URL. This check is only performed if oai parameter is set when calling e.g. getpageinfo_p.xml?actions=oai git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8039 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml |
Tue Nov 15 01:33:54 CET 2011 by orbiter | write info about robots.txt evaluation into getpageinfo_p.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8038 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml, source/de/anomic/crawler/RobotsTxtEntry.java |
Tue Nov 15 01:22:40 CET 2011 by orbiter | - refactoring of getpageinfo_p.xml (moved out of util) - added more logging in getpageinfo_p.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8037 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.html, htroot/ViewFile.html, htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml, htroot/js/Bookmarks.js, htroot/js/IndexCreate.js, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-search.html |
Mon Nov 14 21:19:41 CET 2011 by orbiter | added a link to /api/util/getpageinfo_p.xml as API to crawl start info and to ViewFile.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8035 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartExpert_p.html, htroot/CrawlStartSite_p.html, htroot/ViewFile.html, htroot/js/IndexCreate.js |
Mon Nov 14 16:33:35 CET 2011 by orbiter | - increased stability when opening the robots table - increased stability when deleting tables git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8034 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java, htroot/Tables_p.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/search/Switchboard.java |
Mon Nov 14 16:11:57 CET 2011 by orbiter | refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8033 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java, source/net/yacy/kelondro/util/FileUtils.java |
Mon Nov 14 13:32:15 CET 2011 by orbiter | fix for npe git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8032 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/table/Table.java |
Mon Nov 14 12:37:11 CET 2011 by sixcooler | Trying ImageIO instead of awt-Toolkit for parsing Changed Files: source/net/yacy/document/ImageParser.java |
Mon Nov 14 12:21:36 CET 2011 by sixcooler | not checking for the pid-file Changed Files: .gitignore |
Mon Nov 14 12:20:59 CET 2011 by sixcooler | script for clean copy of URL-tables Changed Files: build.xml, cleanUp.sh |
Mon Nov 14 12:20:33 CET 2011 by sixcooler | custom start-script Changed Files: startYACY.sh |
Mon Nov 14 12:19:58 CET 2011 by sixcooler | custom logging settings Changed Files: yacy.logging |
Mon Nov 14 12:19:27 CET 2011 by sixcooler | some filtering Changed Files: yacy.blue, yacy.stopwords |
Mon Nov 14 12:16:07 CET 2011 by sixcooler | some customize on Memory-Performance-Graph Changed Files: source/net/yacy/kelondro/util/MemoryTracker.java, source/net/yacy/peers/graphics/ProfilingGraph.java |
Mon Nov 14 12:14:07 CET 2011 by sixcooler | weekly rewrite of blobs Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java |
Mon Nov 14 12:11:22 CET 2011 by sixcooler | some more unreserved characters Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Mon Nov 14 12:09:34 CET 2011 by sixcooler | clean offline copy of URL Tables Changed Files: source/de/anomic/data/URLAnalysis.java |
Mon Nov 14 12:08:01 CET 2011 by sixcooler | memory in Perfomance Graph - just like it was in the past Changed Files: htroot/Status.html |
Mon Nov 14 12:06:05 CET 2011 by sixcooler | sixcooler.de in seed-list bootstrap locations Changed Files: defaults/yacy.network.freeworld.unit |
Mon Nov 14 11:54:48 CET 2011 by sixcooler | cutom keep alive strategy Changed Files: source/net/yacy/kelondro/util/G1MemoryStrategy.java |
Mon Nov 14 11:54:03 CET 2011 by sixcooler | G1 Memory Strategy - not used now Changed Files: source/net/yacy/kelondro/util/MemoryControl.java |
Mon Nov 14 11:52:29 CET 2011 by sixcooler | cutom keep alive strategy Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Mon Nov 14 11:46:30 CET 2011 by sixcooler | no force on Memory Request for these parser Changed Files: source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/zipParser.java |
Sat Nov 12 12:26:42 CET 2011 by sixcooler | git testing: Changed Files: .gitignore |
Sat Nov 12 10:25:08 CET 2011 by apfelmaennchen | - small bug fix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8030 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/portalsearch/portalsearch-dev.js, htroot/portalsearch/yacy-portalsearch.js |
Sat Nov 12 00:18:02 CET 2011 by apfelmaennchen | - removed unused code from search widget - added more comments for documentation - ALT key now submits global search - various smaller bug fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8029 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/portalsearch/portalsearch-dev.js, htroot/portalsearch/yacy-portalsearch.js |
Fri Nov 11 00:18:58 CET 2011 by apfelmaennchen | - more improvements for search widget (portalsearch) - added proper error handling - greatly increased robustness - greatly increased usability of navigators - some smaller speed improvements git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8028 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/portalsearch/portalsearch-dev.js, htroot/portalsearch/yacy-portalsearch.js |
Thu Nov 10 12:57:26 CET 2011 by orbiter | typo git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8027 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/de/anomic/crawler/ZURL.java, source/net/yacy/cora/services/federated/solr/SolrShardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrShardingSelection.java, source/net/yacy/search/Switchboard.java |
Thu Nov 10 10:13:58 CET 2011 by apfelmaennchen | - included missing image for portalsearch.tar.gz in build.xml - compressed (minify) yacy-portalsearch.js for better performance - removed language selector, as it doesn't work really well (at least for me) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8026 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml, htroot/portalsearch/portalsearch-dev.js, htroot/portalsearch/yacy-portalsearch.js |
Thu Nov 10 09:54:55 CET 2011 by apfelmaennchen | - portalsearch.js some fixes for paths, when remote loading method is used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8025 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigLiveSearch.html, htroot/portalsearch/yacy-portalsearch.js |
Thu Nov 10 00:37:35 CET 2011 by apfelmaennchen | - some smaller changes to search widget git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8024 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/portalsearch/yacy-portalsearch.css, htroot/portalsearch/yacy-portalsearch.js |
Wed Nov 09 16:54:01 CET 2011 by orbiter | fixed directDocByURL (has now effect if switched off) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8022 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java |
Wed Nov 09 16:40:14 CET 2011 by orbiter | fixed urls to media content during indexing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8021 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/ResultURLs.java, source/net/yacy/document/TextParser.java, source/net/yacy/search/Switchboard.java |
Wed Nov 09 00:01:22 CET 2011 by apfelmaennchen | fix for http://bugs.yacy.net/view.php?id=68 as well as for a far more serious bug in navigator handling in the portal search widget. Navigators are now quite usable, but the GUI has still some flaws... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8019 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/portalheader.template, htroot/jquery/css/yacy-portalsearch.css, htroot/jquery/css/yacy-ymarks.css, htroot/jquery/js/yacy-portalsearch.js |
Tue Nov 08 16:38:08 CET 2011 by orbiter | - added a cache for active crawl profiles to the crawl switchboard - moved the domain cache for domain counter from the crawl switchboard to the crawl profiles. the crawl domain counter is now therefore relative for each crawl start, not for the whole crawler. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8018 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/net/yacy/search/Switchboard.java |
Tue Nov 08 13:02:22 CET 2011 by orbiter | normalization of url using urlencoding/decoding git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8017 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, source/de/anomic/crawler/RobotsTxtParser.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/UTF8.java |
Tue Nov 08 12:13:29 CET 2011 by apfelmaennchen | fix for http://bugs.yacy.net/view.php?id=69 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8015 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/jquery/css/yacy-portalsearch.css, htroot/jquery/css/yacy-ymarks.css, htroot/jquery/js/yacy-portalsearch.js |
Fri Nov 04 00:44:49 CET 2011 by suessthomas | The IP address of the YaCy-Demo portal added to Whitelist. This is only a temporary workaround. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8013 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.freeworld.unit, defaults/yacy.network.metager.unit |
Thu Nov 03 15:41:38 CET 2011 by orbiter | stability bugfix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8012 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/table/Table.java |
Thu Nov 03 15:34:58 CET 2011 by orbiter | stability bugfixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8011 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/NewsQueue.java, source/net/yacy/visualization/RasterPlotter.java |
Tue Nov 01 23:49:14 CET 2011 by orbiter | fix for link in Supporter and Suftipps page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8010 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Supporter.java, htroot/Surftips.java, source/net/yacy/kelondro/index/Row.java |
Sun Oct 30 02:09:38 CET 2011 by orbiter | - updated opensearchdescription text and icon - removed automatic setting of maxitems during search (can be set now elsewhere) - updated RSSMessage.java git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8009 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/opensearchdescription.xml, htroot/yacysearch.java, source/net/yacy/cora/document/RSSMessage.java |
Sat Oct 29 11:22:03 CEST 2011 by orbiter | set default number of search results in ConfigPortal.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8008 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java |
Sat Oct 29 10:45:43 CEST 2011 by orbiter | fix for npe git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8007 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.java, source/net/yacy/peers/NewsQueue.java |
Sat Oct 29 09:28:24 CEST 2011 by orbiter | fix for npe git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8006 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigPortal.java |
Fri Oct 28 01:21:39 CEST 2011 by orbiter | suppressing size if size unknown git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8005 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js |
Wed Oct 26 12:07:16 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=62 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8004 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/NoticedURL.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/table/Table.java |
Sun Oct 23 21:53:51 CEST 2011 by lotus | Win installer: remove Berlios redirect to updated JRE, link is now hardcoded again, JRE update git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8003 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/download.php, build.nsi |
Sat Oct 22 18:37:56 CEST 2011 by low012 | *) set maxlength of input field for country code filter to value > default text length (old value caused warning in Opera) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8002 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartExpert_p.html |
Fri Oct 21 01:06:49 CEST 2011 by orbiter | performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8001 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/idx.java, htroot/yacysearch.java, source/de/anomic/http/server/HTTPDemon.java |
Wed Oct 19 01:38:19 CEST 2011 by orbiter | enhanced table method (used by almost all yacy api interfaces) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8000 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/yacy.java |
Thu Oct 13 02:34:52 CEST 2011 by orbiter | new snippet strategy for search hits in metadata: show beginning of text instead of hit position git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7999 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/search/snippet/TextSnippet.java |
Thu Oct 13 00:59:58 CEST 2011 by orbiter | BOOSTED the image search function. The result page now shows the images as embedded image link from the original source and not from the built-in image buffering and re-sizing servlet. The result is shown much faster now not because YaCy does not need to re-size the images but for a very strange other reason: because of RFC specification (http://tools.ietf.org/html/rfc2616#section-8.1.4) a browser does not open more than two connections to the same server at the same time. If the YaCy image servlet is used, then the target host is the YaCy host for all images and that prevents a parallel computation of the image loading. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7998 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacysearchitem.html |
Thu Oct 13 00:26:48 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=59 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7997 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/server/serverObjects.java, source/net/yacy/cora/protocol/RequestHeader.java |
Mon Oct 10 17:23:37 CEST 2011 by hermens | make sure myself-target is always inserted at its appropriate position this was previously omitted if the own peer should have been the first target or the peer was the last peer before the rotation to AAAAAAAAAAAA git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7996 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/dht/PeerSelection.java |
Mon Oct 10 16:35:03 CEST 2011 by hermens | added sender side prevention of rwi flooding as mentioned in SVN 7993 saves memory and speeds up enqueueContainers by limiting the size of transfer.Chunk saves network bandwidth by not transmitting RWIs that would get discarded at the target anyway git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7995 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/peers/dht/Transmission.java |
Mon Oct 10 11:46:38 CEST 2011 by orbiter | enhanced exported row parsing during row import this affects the search and dht receive speed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7994 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/search/Switchboard.java |
Mon Oct 10 03:10:49 CEST 2011 by orbiter | added a protection against rwi flooding git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7993 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/transferRWI.java |
Sat Oct 08 01:23:49 CEST 2011 by orbiter | fix for thread dump git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7992 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Sat Oct 08 00:57:20 CEST 2011 by orbiter | added new scripts to bin in build git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7991 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Sat Oct 08 00:53:14 CEST 2011 by orbiter | update to ThreadDump analysis: freerunner and thread state recognition git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7990 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Fri Oct 07 01:35:14 CEST 2011 by orbiter | another performance hack git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7989 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/order/NaturalOrder.java |
Fri Oct 07 01:31:04 CEST 2011 by orbiter | performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7988 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/NaturalOrder.java |
Fri Sep 30 15:39:01 CEST 2011 by orbiter | added robots tag parser to solr scheme git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7986 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java |
Fri Sep 30 12:27:14 CEST 2011 by orbiter | - fix for incomplete clear() in balancer - renamed Parser Errors to Rejected URLs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7984 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexCreateParserErrors_p.html, htroot/env/templates/submenuCrawlMonitor.template, locales/de.lng, source/de/anomic/crawler/Balancer.java |
Thu Sep 29 17:30:15 CEST 2011 by orbiter | show IP of crawled host and country in CrawlResults git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7981 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlResults.html, htroot/CrawlResults.java |
Thu Sep 29 14:53:55 CEST 2011 by f1ori | * fix urlproxy for urls containing dolar signs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7979 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Wed Sep 28 23:18:49 CEST 2011 by orbiter | added dump check script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7978 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/dumpcheck.sh |
Wed Sep 28 23:16:40 CEST 2011 by orbiter | performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7977 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/logging/ThreadDump.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/peers/yacySeed.java |
Tue Sep 27 17:26:14 CEST 2011 by orbiter | added a new feature to MultiProtocolURIs to get the locale for each url: This is done using a new library InetAddressLocator.jar which is NOT added by default to YaCy because it is very old and with that library we will never get a debian package. However, some people want that functionality and it can be made available if the library is taken from http://javainetlocator.sourceforge.net/ and placed into the /lib directory where it will be found using reflection. The new feature will be used to extend the crawler steering. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7975 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/plugin/ClassProvider.java, source/net/yacy/cora/protocol/Domains.java |
Sun Sep 25 18:07:22 CEST 2011 by low012 | *) this should fix a bug in snippet creation (also cleaned up a little bit) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7972 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/TextSnippet.java |
Sun Sep 25 15:16:52 CEST 2011 by low012 | *) added comments *) minor refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7971 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/CharacterCoding.java |
Sat Sep 24 03:58:54 CEST 2011 by orbiter | - removed some warinings - removed a dead update location git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7970 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.freeworld.unit, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/ResultFetcher.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/document/WordTokenizer.java |
Thu Sep 22 01:33:44 CEST 2011 by orbiter | added script that can be used to delete a single url from the index call: bin/deleteurl.sh <url> git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7969 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/deleteurl.sh, htroot/IndexControlURLs_p.java |
Wed Sep 21 17:08:05 CEST 2011 by orbiter | - more tolerance against failure of table opening - more connections for solrj git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7968 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/MetadataRepository.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/kelondro/table/SplitTable.java |
Wed Sep 21 13:01:01 CEST 2011 by orbiter | fix in result count display git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7967 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacysearch.js |
Wed Sep 21 12:43:08 CEST 2011 by orbiter | enhanced search result timing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7966 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/ResultFetcher.java |
Tue Sep 20 21:41:44 CEST 2011 by orbiter | - add number of search results to end of rss and json output to reflect latest status of retrieval - distinguish search access with different verify state in access of search cache git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7965 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml, source/de/anomic/search/QueryParams.java, source/de/anomic/server/serverObjects.java, source/net/yacy/cora/services/federated/yacy/CacheStrategy.java |
Tue Sep 20 17:02:15 CEST 2011 by f1ori | * fix urls containing a port number in urlproxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7964 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Tue Sep 20 16:54:14 CEST 2011 by f1ori | * pass cookies to Server Side Includes * User.html a bit more usable git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7963 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/User.html, htroot/User.java, source/de/anomic/data/UserDB.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/ServerSideIncludes.java |
Sun Sep 18 18:59:01 CEST 2011 by lotus | fix: follow link target setting on image search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7962 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.html, htroot/yacysearchitem.java |
Fri Sep 16 12:00:51 CEST 2011 by orbiter | performance hack for search process git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7961 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/RankingProcess.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java |
Fri Sep 16 01:26:09 CEST 2011 by orbiter | fixed missing menu entries in access tracker git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7960 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/submenuAccessTracker.template |
Fri Sep 16 00:40:01 CEST 2011 by orbiter | yet another cache strategy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7959 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/yacy/CacheStrategy.java |
Thu Sep 15 13:17:38 CEST 2011 by orbiter | ensure termination of pdf parser to avoid deadlocking of other processes during search result preparation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7958 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/pdfParser.java |
Thu Sep 15 13:17:02 CEST 2011 by orbiter | bugfixes and performance hacks for tabe index git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7957 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java |
Thu Sep 15 01:34:05 CEST 2011 by orbiter | performance hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7955 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/transferRWI.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java |
Wed Sep 14 23:19:02 CEST 2011 by orbiter | synchronization enhancements git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7954 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/date/GenericFormatter.java |
Wed Sep 14 22:11:27 CEST 2011 by orbiter | patches git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7953 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyRelease.java, source/net/yacy/kelondro/index/Row.java |
Wed Sep 14 22:08:59 CEST 2011 by orbiter | migration to solrj 3.4.0 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7952 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/apache-solr-solrj-3.4.0.License, lib/apache-solr-solrj-3.4.0.jar, lib/dependencies.txt |
Wed Sep 14 11:15:09 CEST 2011 by orbiter | added keyboard-based search result page navigation: - page-up or tab switches to next search result page - page-down switches to previous search result page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7950 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.html |
Wed Sep 14 11:05:06 CEST 2011 by orbiter | - style change for search box: larger font, selected by default - style change for search results: by default no parser, size, image info git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7949 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/compare_yacy.html, htroot/env/base.css, htroot/index.html, htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacysearch.html |
Wed Sep 14 02:33:08 CEST 2011 by orbiter | fix for image name git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7947 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Banner.java |
Wed Sep 14 02:03:33 CEST 2011 by orbiter | fresh look for yacy icons git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7946 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/base.css, htroot/env/grafics/YaCyLogo_120ppi.png, htroot/env/grafics/YaCyLogo_60ppi.png, htroot/env/grafics/yacy.png |
Wed Sep 14 01:34:36 CEST 2011 by orbiter | replaced old-style favicon git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7945 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/favicon.bmp, htroot/favicon.ico, htroot/favicon.png |
Tue Sep 13 20:46:11 CEST 2011 by orbiter | fix for problem with relocation of network git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7944 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java |
Sat Sep 10 01:00:45 CEST 2011 by orbiter | protection against OOM cases in image parser. See also bugs.yacy.net/view.php?id=54 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7942 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/images/genericImageParser.java |
Thu Sep 08 21:13:19 CEST 2011 by orbiter | try to fix bug http://bugs.yacy.net/view.php?id=26 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7941 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeed.java |
Thu Sep 08 20:47:00 CEST 2011 by orbiter | try to avoid the unresolved pattern in search result git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7940 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacysearch.js, htroot/yacysearchlatestinfo.java |
Thu Sep 08 14:54:10 CEST 2011 by orbiter | fix for clear commands git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7939 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/clearall.sh, bin/clearindex.sh |
Thu Sep 08 14:23:55 CEST 2011 by orbiter | - fix for wrong entries in NOLOAD indexing queue (that caused that urls had been only indexed based on their url and not loaded) - patch for better urls to solr admin interface git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7938 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java |
Thu Sep 08 00:15:01 CEST 2011 by orbiter | added synchronization in ReferenceContainer and logging for shrinking git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7937 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java |
Wed Sep 07 23:47:54 CEST 2011 by orbiter | added reference limitation to IndexControlRWIs_p.html servlet git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7936 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java |
Wed Sep 07 20:55:16 CEST 2011 by sixcooler | refactored stuff from last commit to ReferenceContainer see: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3353&p=23163#p23163 the limiting of references is disabled per default to enable this set yacy.conf - index.maxReferences to a value of e.g. 100000 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7935 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java |
Wed Sep 07 15:08:06 CEST 2011 by sixcooler | limit references per blob & term to the 100.000 youngest git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7934 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java |
Wed Sep 07 13:42:06 CEST 2011 by orbiter | added a fast mass-deletion method git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7933 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/RowSet.java |
Wed Sep 07 12:09:35 CEST 2011 by orbiter | added another file tool class to yacy-cora git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7932 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/Files.java |
Wed Sep 07 11:07:40 CEST 2011 by orbiter | fix for solr url git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7930 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java |
Mon Sep 05 14:21:25 CEST 2011 by f1ori | fixes size of document in case the server doesn't give the size in the header git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7929 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/Response.java |
Mon Sep 05 02:26:27 CEST 2011 by orbiter | reverting SVN 7863; the synchronization was needed and no synchronization causes repeated DNS lookup for the same hosts git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7928 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/Domains.java |
Mon Sep 05 02:22:21 CEST 2011 by orbiter | ensure termination of shrink() git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7927 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Mon Sep 05 02:09:48 CEST 2011 by orbiter | replaced catching of Exception with Throwable, which catches also Errors git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7926 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java |
Mon Sep 05 01:28:15 CEST 2011 by orbiter | switched off logging of org.apache.http which was suddenly switched on by default (??) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7925 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/yacy.java, yacy.logging |
Mon Sep 05 01:27:12 CEST 2011 by orbiter | moved merger object from Segment to IndexCell to enable a correct shutdown sequence. This solves a bug where yacy cannot be shut down during an index merge that appears during the shutdown phase. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7924 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Segment.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java |
Sat Sep 03 09:10:47 CEST 2011 by orbiter | fix for javascript error during search (not ready yet) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7923 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacysearch.js |
Sat Sep 03 01:43:04 CEST 2011 by orbiter | added more libraries that are needed by solrj git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7922 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/dependencies.txt, lib/geronimo-stax-api_1.0_spec-1.0.1.License, lib/geronimo-stax-api_1.0_spec-1.0.1.jar, lib/jcl-over-slf4j-1.6.1.jar, lib/wstx-asl-3.2.7.jar |
Fri Sep 02 18:50:05 CEST 2011 by sixcooler | fixed a bug from 7856, where Snippet returned an error by mistake when Metadata was found git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7921 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/TextSnippet.java |
Thu Sep 01 18:08:33 CEST 2011 by orbiter | a bit protection against search result bugs in interactive search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7920 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js |
Thu Sep 01 18:02:15 CEST 2011 by orbiter | be more liberal when evaluating search results. This may cause that it is possible to fraud content on fresh peers, but that is better than looong waiting times for the evaluation of every link which causes that everybody rejects YaCy as 'too slow'. But this is only because of the high standards that YaCy sets to itself. If we are able to gain more users by lowering the standard, then that is useful. The option to set that flag to verify each link is still there. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7918 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init |
Thu Sep 01 15:13:21 CEST 2011 by orbiter | protection against starting of a (second) yacy peer while another one is already running on the same port git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7917 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/yacy.java |
Thu Sep 01 12:37:14 CEST 2011 by orbiter | removed unnecessary warning git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7916 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java |
Thu Sep 01 12:25:27 CEST 2011 by orbiter | small fix for count: overXX includes the count git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7915 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java |
Thu Sep 01 12:23:22 CEST 2011 by orbiter | enhanced termlist git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7914 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java, htroot/api/termlist_p.xml |
Thu Sep 01 12:14:17 CEST 2011 by orbiter | added new word recommendation library in DictionaryLoader_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7913 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/WordCache.java |
Wed Aug 31 18:02:06 CEST 2011 by orbiter | bugfixes in html parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7912 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/repository/LoaderDispatcher.java, startYACY.sh |
Mon Aug 29 11:16:41 CEST 2011 by orbiter | removed test output git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7911 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewFile.java |
Mon Aug 29 00:59:19 CEST 2011 by orbiter | fixed a parsing bug by reverting SVN 7766 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7910 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/TransformerWriter.java |
Sun Aug 28 16:24:09 CEST 2011 by low012 | *) Changed action to "" instead of "yacysearch.html". This should not do any harm, but helps a lot if the page is accessed not by its original name but by a different name which can be done by adding a symbolic link to the file system of the peer. (See http://www.yacy-forum.org/viewtopic.php?f=2&t=464) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7909 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.html |
Sun Aug 28 15:32:42 CEST 2011 by low012 | *) replacing one ugly hack with an extended ugly hack ;-) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7908 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/RSSReader.java |
Sun Aug 28 14:59:11 CEST 2011 by low012 | *) should fix some problems with RSS Importer (see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3253) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7907 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/RSSReader.java |
Sun Aug 28 03:14:58 CEST 2011 by low012 | *) changed links to be underlined again since lots of links were not obvious anymore *) added SVN properties git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7906 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/base.css |
Sat Aug 27 17:23:24 CEST 2011 by sixcooler | Generation Memory Strategy fine tuning add some log-output in termlist_p git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7904 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java |
Fri Aug 26 00:32:15 CEST 2011 by sixcooler | Generation Memory Strategy fine tuning: - some more optimism on requests of unknown values - avoid a premature value of 0 byte available git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7903 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/kelondro/util/MemoryStrategy.java |
Thu Aug 25 23:12:31 CEST 2011 by orbiter | limitation of reference evaluation (protection against crawler pits) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7902 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/WebStructureGraph.java |
Thu Aug 25 21:13:45 CEST 2011 by orbiter | - added memory computation to termlist_p.xml - added option to delete terms in termlist_p.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7901 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java, htroot/api/termlist_p.xml, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java |
Thu Aug 25 20:29:30 CEST 2011 by orbiter | fix for term iteration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7900 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/ranking/Rating.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java |
Thu Aug 25 18:49:20 CEST 2011 by orbiter | added a servlet that produces a list of term hashes that appear more than 10000 times see /api/termlist_p.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7898 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/termlist_p.java, htroot/api/termlist_p.xml |
Thu Aug 25 17:52:25 CEST 2011 by orbiter | - bugfixes in html parser - new fields in solr - extended file viewer to debug parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7897 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, htroot/ViewFile.html, htroot/ViewFile.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/TransformerWriter.java |
Thu Aug 25 01:08:28 CEST 2011 by orbiter | refactoring/better naming of methods and classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7895 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java |
Wed Aug 24 14:12:48 CEST 2011 by orbiter | fix in search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7894 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/ResultFetcher.java |
Wed Aug 24 14:07:53 CEST 2011 by sixcooler | - not doing merge-jobs while short on Memory - using configuration-values of crawling-max-filesize also for snippetfetching and loading files into Index git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7893 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/repository/LoaderDispatcher.java |
Wed Aug 24 12:27:19 CEST 2011 by orbiter | enhanced sorting speed (affects all DB operations) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7892 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java |
Tue Aug 23 22:23:40 CEST 2011 by orbiter | added iterable implementation in KeyList git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7891 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/KeyList.java |
Tue Aug 23 00:33:08 CEST 2011 by orbiter | added new score comparator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7888 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/ranking/ScoreComparator.java |
Mon Aug 22 19:58:23 CEST 2011 by sixcooler | almost revert changes in this class of 7880 and 7882 since MemoryControl does handle negative value requests git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7887 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDemon.java |
Mon Aug 22 19:45:27 CEST 2011 by sixcooler | do not look at external dtd, cause this make this reader stay forewer(?) on on faulty dtd-locations git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7885 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/RSSReader.java |
Mon Aug 22 12:19:15 CEST 2011 by orbiter | by default show only domain navigator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7884 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.html |
Thu Aug 18 00:24:17 CEST 2011 by orbiter | - added a short memory status simulation mode - added a button in PerformanceMemory_p.html to set the simulated short memory status - bugfix: added a missing lowercase in KeyList - better concurrency in loader dispatcher git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7883 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/repository/LoaderDispatcher.java |
Mon Aug 15 13:07:03 CEST 2011 by orbiter | addon to svn 7880 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7882 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/kelondro/util/MemoryControl.java |
Sun Aug 14 14:28:28 CEST 2011 by sixcooler | ignore cookies in httpclient per default disable cookiestore,cause the default one caused segfaults on my peers this does not harm use of cookies via YaCy as proxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7881 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Sat Aug 13 03:04:37 CEST 2011 by sixcooler | since many POST-requests come as gzip they report a contentlength of -1 request memory of -1 * 3 look useless to me so I added some megs to it - even correct report of contentlength should not be harmed by this git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7880 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDemon.java |
Sat Aug 13 02:59:59 CEST 2011 by sixcooler | do some extra sleep while running low on memory (1 sec. per outofmemoryCycle) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7879 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/workflow/AbstractBusyThread.java |
Fri Aug 12 20:07:46 CEST 2011 by orbiter | select the search window on focus so its easy to type in another query git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7878 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacyinteractive.html |
Fri Aug 12 19:51:30 CEST 2011 by sixcooler | using GzipDecompressingEntity from httpclient instead of our own (was just fixed there in httpclient-4.1.2 and does a proper job) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7877 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/GzipResponseInterceptor.java |
Fri Aug 12 19:13:34 CEST 2011 by orbiter | better interaction git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7875 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacyinteractive.html |
Fri Aug 12 18:09:40 CEST 2011 by orbiter | added a 'loading...' message git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7874 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacyinteractive.html |
Fri Aug 12 16:42:20 CEST 2011 by orbiter | switch off real-time search if index is large git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7873 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacyinteractive.html, htroot/yacyinteractive.java |
Fri Aug 12 15:27:11 CEST 2011 by sixcooler | try better handling of corrupt blobs @developer: please revert if I'm wrong see http://forum.yacy-websuche.de/viewtopic.php?f=8&t=3334 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7872 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/ReferenceIterator.java |
Fri Aug 12 15:21:18 CEST 2011 by orbiter | - bugfix in ServerScannerList - speed up of generation of scanner list avoiding forced dns lookup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7871 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ServerScannerList.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java |
Fri Aug 12 14:25:45 CEST 2011 by orbiter | better integration of server list in interactive search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7870 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacyinteractive.java, source/net/yacy/cora/protocol/Scanner.java |
Fri Aug 12 12:31:01 CEST 2011 by orbiter | added a serlvet that shows all scanned servers inside of the yacyinteractive search page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7869 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ServerScannerList.html, htroot/ServerScannerList.java, htroot/yacyinteractive.html |
Fri Aug 12 10:06:21 CEST 2011 by orbiter | do not sort concrrently the same array git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7868 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/RowCollection.java |
Thu Aug 11 23:54:27 CEST 2011 by orbiter | same values for Xmx and Xms: memory extension may be difficult if the OS has not the remaining memory available and may kill the jvm. If the memory is reserved at the start but never used the OS may handle that as well and leave non-used space in swap area (and never swap) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7867 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/PerformanceQueues_p.java |
Thu Aug 11 23:37:35 CEST 2011 by orbiter | #cccamp11 maintenance fix: anons may find up to 1000 items in interactive search (was: 100) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7866 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacyinteractive.java, htroot/yacysearch.java, source/de/anomic/search/QueryParams.java |
Thu Aug 11 23:07:52 CEST 2011 by sixcooler | encapsulate potential expensive objects in TextSnippet to allow GC them asap this reduces chance of OOMs at massive search & snippet-fetching git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7865 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/TextSnippet.java, source/net/yacy/document/WordTokenizer.java |
Thu Aug 11 09:31:35 CEST 2011 by cominch | the form value was not correctly interpreted git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7864 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Bookmarks.java |
Wed Aug 03 21:42:18 CEST 2011 by orbiter | removed synchronization in DNS resolve (solves a problem when loading snippets but in the past concurrent dns requests also caused deadlocks. but this is many years ago and we will give it another try) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7863 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/Domains.java |
Wed Aug 03 20:32:11 CEST 2011 by orbiter | performance hacks for seed generation (because thread dumps showed multiple occurrences at these code points) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7861 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/tools/gzip.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/kelondro/util/MapTools.java |
Tue Aug 02 17:52:33 CEST 2011 by sixcooler | early freeing resources on deleting index reference if search-verification fails (aka Switchboard.cleanupJob) doing same thingy on other methods of touched files as well git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7860 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/WorkTables.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/util/BDecoder.java |
Tue Aug 02 01:35:24 CEST 2011 by sixcooler | finishing up my commits (7855-7858) which could be helpful for not declaring inside loops (helps GC of some VMs) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7859 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/net/yacy/document/SnippetExtractor.java |
Tue Aug 02 01:32:58 CEST 2011 by sixcooler | throwing an exception again in FileUtils.copy(reader, writer) OOMs could occour here and should not be ignored git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7858 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/util/FileUtils.java |
Tue Aug 02 01:32:29 CEST 2011 by sixcooler | stoping thread for fetching searchresults if running short on memory - in most cases at least one thread stays alive for getting the results - fewer threads should do the work with less resouces, but much slower then git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7857 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/ResultFetcher.java |
Tue Aug 02 01:31:08 CEST 2011 by sixcooler | less byte-arrays of response-content, less byte-array <-> stream conversation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7856 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/document/TextParser.java |
Tue Jul 26 03:05:28 CEST 2011 by sixcooler | correct value @ low memory warning see: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3317 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7854 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Status.java |
Mon Jul 25 02:48:29 CEST 2011 by sixcooler | bump to httpcore-4.1.2: a bugfixrelease git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7853 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/httpcore-4.1.2.License, lib/httpcore-4.1.2.jar |
Tue Jul 19 14:48:50 CEST 2011 by sixcooler | Runtime.maxMemory() DOES change @ runtime: I wondered getting Total-ram > Max-ram and MemoryControl.available() < 0 MemoryControl.available() < 0 causes some errors where its value is used for dimension of buffers for eg. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7852 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.java, htroot/Status.java, htroot/api/status_p.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/yacy.java |
Tue Jul 19 14:04:40 CEST 2011 by f1ori | * revert parts of previous commit, because it breaks the trickle-feature git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7851 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Tue Jul 19 13:47:53 CEST 2011 by f1ori | * use gzip-encoding in more cases * send Expire-Header for static content * should improve webserver-performance for slow connections * fixes #37 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7850 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java |
Sun Jul 17 19:55:08 CEST 2011 by orbiter | added another memory protection hack git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7849 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/ContentScraper.java |
Sun Jul 17 10:39:39 CEST 2011 by orbiter | more protection for the cleanup thread git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7848 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Sun Jul 17 02:02:42 CEST 2011 by orbiter | changed the way how the index cache is flushed: do not flush when a put was made because that could cause that many put calls synchronize for a long time when the dump or a merge is performed. Instead a watchdog thread is doing the dump and therefore puts cannot block any more which is good when a put happens during a search result preparation. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7847 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/rwi/IndexCell.java |
Sun Jul 17 01:17:19 CEST 2011 by orbiter | do not allocate a StringBuilder object in case that there is not enough memory for that git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7846 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/AbstractScraper.java |
Sat Jul 16 21:21:39 CEST 2011 by orbiter | do not sort arrays again if arrays are already sorted git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7845 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/Array.java, source/net/yacy/kelondro/index/RowCollection.java |
Sat Jul 16 21:13:30 CEST 2011 by orbiter | - refactoring - do not start worker threads in Array class if concurrency is not used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7844 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/Array.java, source/net/yacy/cora/storage/Sortable.java, source/net/yacy/cora/storage/UpDownLatch.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/yacy.java |
Sat Jul 16 13:54:47 CEST 2011 by orbiter | disabling concurrency in new sort since that is not working yet correctly git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7843 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/order/Array.java |
Fri Jul 15 11:14:21 CEST 2011 by sixcooler | smaler ping-entry @ ProfilingGraph git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7841 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/ProfilingGraph.java |
Thu Jul 14 23:42:30 CEST 2011 by orbiter | code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7839 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java |
Thu Jul 14 13:00:38 CEST 2011 by orbiter | added request line in http which can support better debugging git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7838 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverCore.java |
Thu Jul 14 05:25:57 CEST 2011 by orbiter | because newbies are scared about the memory consumption in the performance graph and arguments about high memory consumption according to bad knowledge about java garbage collection techniques, the memory display had been removed from the performance graph shown on the Status.html page. The memory graph can still be seen on the Performance page where the memory graph is just like it was. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7836 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceGraph.java, htroot/Status.html, source/de/anomic/search/SearchEventCache.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/net/yacy/visualization/ChartPlotter.java |
Tue Jul 12 09:44:23 CEST 2011 by cominch | do not replace malformed or invalid URLs in urlproxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7835 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Wed Jul 06 16:29:17 CEST 2011 by orbiter | update to eclipse class path environmen git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7834 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath |
Tue Jul 05 11:21:30 CEST 2011 by orbiter | fix for solr auth git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7833 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/solr/SolrChardingConnector.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Tue Jul 05 08:54:05 CEST 2011 by orbiter | no error message about missing browser in headless environments git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7832 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/yacy.java |
Mon Jul 04 19:04:49 CEST 2011 by orbiter | temporary adding the old httpclient-3.1 again because the solrj classes need them. should be removed as soon solrj supports httpclient-4 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7831 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/commons-httpclient-3.1.jar, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Mon Jul 04 18:53:18 CEST 2011 by orbiter | fixed bad query git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7830 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Mon Jul 04 12:48:01 CEST 2011 by orbiter | docfix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7828 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list |
Mon Jul 04 00:59:43 CEST 2011 by orbiter | fix for solr scheme list definition git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7826 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list |
Mon Jul 04 00:55:36 CEST 2011 by orbiter | fixed solr scheme table view git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7825 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexFederated_p.java |
Sun Jul 03 23:40:12 CEST 2011 by orbiter | same for debian as for latest commit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7824 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/yacyInit.m4 |
Sun Jul 03 20:13:37 CEST 2011 by orbiter | replaced method to call getUsableSpace using reflection with direct call since we now use java 1.6 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7821 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/ResourceObserver.java |
Sun Jul 03 08:40:05 CEST 2011 by orbiter | - added counting of links with noindex tag for solr index - bugfixes for solr index git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7820 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, htroot/IndexFederated_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/document/Document.java |
Sat Jul 02 23:10:13 CEST 2011 by orbiter | replaced commons-io v1.4 with v2.0.1 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7818 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/commons-io-2.0.1.jar, lib/commons-io-2.0.1.license |
Sat Jul 02 22:52:33 CEST 2011 by orbiter | replaced fontbox, jempbox, pdfbox v 1.5 with v1.6 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7817 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.6.0.jar, lib/fontbox-1.6.License, lib/jempbox-1.6.0.License, lib/jempbox-1.6.0.jar, lib/pdfbox-1.6.0.jar, lib/pdfbox-1.6.License |
Sat Jul 02 19:23:37 CEST 2011 by lotus | hint for proxy scraping git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7816 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ProxyIndexingMonitor_p.html |
Sat Jul 02 19:16:00 CEST 2011 by lotus | use proxy accounts by default for more security http://bugs.yacy.net/view.php?id=45 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7815 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init |
Sat Jul 02 02:09:49 CEST 2011 by orbiter | added default configuration in ConfigurationSet in case of new values git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7814 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/cora/storage/ConfigurationSet.java |
Sat Jul 02 00:50:46 CEST 2011 by orbiter | added XX:MaxPermSize to debian and mac start scripts git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7813 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, addon/yacyInit.m4 |
Fri Jul 01 18:38:01 CEST 2011 by orbiter | added parsing of canonical link element to html parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7812 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/document/parser/html/ContentScraper.java |
Thu Jun 30 17:44:06 CEST 2011 by orbiter | added "-XX:MaxPermSize=256m" to start script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7810 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: startYACY.sh |
Thu Jun 30 17:37:14 CEST 2011 by orbiter | fixed Semaphore handling in case of interruptions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7809 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IODispatcher.java |
Wed Jun 29 17:35:45 CEST 2011 by orbiter | added parser for such files like the new solr.key.list it parses text files with the following syntax: - all lines beginning with '##' are comments - all non-empty lines not beginning with '#' are keyword lines - all lines beginning with '#' and where the second character is not '#' are commented-out keyword lines git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7808 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/ConfigurationSet.java |
Wed Jun 29 17:33:27 CEST 2011 by orbiter | added keyword list for solr index attributes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7807 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/solr.keys.list |
Wed Jun 29 17:30:24 CEST 2011 by f1ori | * navigation bar for filetype constraints javascript interpreted backslashes from urlmask as escaping and didn't forward them to yacy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7806 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/QueryParams.java |
Wed Jun 29 11:32:02 CEST 2011 by f1ori | * fix url proxy for relative links and chromium git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7805 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Wed Jun 29 11:32:00 CEST 2011 by f1ori | small pi link on index page to administration pages git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7804 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/index.html, htroot/index.java |
Mon Jun 27 01:14:02 CEST 2011 by orbiter | fix for channel authorization git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7803 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/feed.java, source/de/anomic/yacy/yacyChannel.java |
Sun Jun 26 23:45:04 CEST 2011 by orbiter | removed the semaphores from the cache dump process because I believe some of the semaphores may be lost somewhere which then causes that the cache is never flushed and then the peer dies from a OOM. The re-introduced synchronization may not be the best solution but should ensure that the caches are flushed. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7802 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Sun Jun 26 23:17:02 CEST 2011 by orbiter | faster appearance of search result page (but complete search time is the same) this was inspired by http://bugs.yacy.net/view.php?id=37 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7801 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java, htroot/yacysearchlatestinfo.java, source/de/anomic/search/ResultFetcher.java |
Sun Jun 26 23:15:13 CEST 2011 by orbiter | enhanced template engine git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7800 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/TemplateEngine.java |
Sat Jun 25 12:15:42 CEST 2011 by low012 | *) trying to at least fix symptoms of http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3293#p22791 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7799 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java |
Fri Jun 24 10:27:36 CEST 2011 by orbiter | added more info to DNS cache in /PerformanceMemory_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7798 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, source/net/yacy/cora/protocol/Domains.java |
Fri Jun 24 02:24:00 CEST 2011 by orbiter | code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7797 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/ReferenceContainerCache.java |
Fri Jun 24 02:22:57 CEST 2011 by orbiter | added new navigation to search trailer json and xml files which causes that these navigation is also available in the search widget git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7796 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml |
Thu Jun 23 17:39:52 CEST 2011 by orbiter | added a file type navigator added a protocol navigator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7795 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/yacysearch.html, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/migration.java |
Wed Jun 22 12:19:32 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=41 added another RSS channel "PROXY". the rss feed for peer news filters this channel if there is not an authorized access on that channel git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7792 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Status.html, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyChannel.java |
Wed Jun 22 12:04:30 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=42 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7791 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/SettingsAck_p.html, htroot/SettingsAck_p.java |
Wed Jun 22 01:10:50 CEST 2011 by orbiter | performance hacks: more pre-allocated StringBuilder git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7790 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/yacy/dht/Dispatcher.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/net/yacy/kelondro/table/Table.java |
Tue Jun 21 00:53:43 CEST 2011 by orbiter | fixed warning git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7789 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Tue Jun 21 00:33:45 CEST 2011 by orbiter | added @Deprecated git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7788 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/NonClosingInputStreamEntity.java |
Tue Jun 21 00:29:45 CEST 2011 by orbiter | enhanced content scraper (should discover urls much faster in case of very large plain texts) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7787 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/ContentScraper.java |
Sun Jun 19 15:34:55 CEST 2011 by orbiter | disabled assert in Base64Order which eats away too much performance during testing with -l git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7786 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/order/Base64Order.java |
Sun Jun 19 15:31:10 CEST 2011 by orbiter | enhanced identificator: using AtomicInteger for counter git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7785 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/language/Identificator.java |
Sun Jun 19 15:15:01 CEST 2011 by orbiter | unwrapping of call depth (one call less for UTF8.String) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7784 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/UTF8.java |
Sun Jun 19 15:02:06 CEST 2011 by orbiter | enhanced template engine git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7783 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/TemplateEngine.java, source/net/yacy/kelondro/util/ByteBuffer.java |
Sun Jun 19 14:38:06 CEST 2011 by orbiter | enhanced UTCDiffString git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7782 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/date/GenericFormatter.java |
Wed Jun 15 20:57:04 CEST 2011 by lotus | display cache hit/miss values in correct column git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7781 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.html |
Wed Jun 15 11:14:51 CEST 2011 by f1ori | always shutdown log, fixes zombie processes in init stop script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7780 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/yacy.java |
Sat Jun 11 23:22:08 CEST 2011 by suessthomas | The Java-XMS and Xmx values for the target of "run" (run YaCy) inserted. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7777 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Mon Jun 06 00:58:17 CEST 2011 by sixcooler | prevent getting a yacySeed from zero-length-hash-string by chance (for eg.: proxy-crawls got displayed as initiated by some other peer) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7776 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeedDB.java |
Mon Jun 06 00:48:21 CEST 2011 by orbiter | update on options for interface graphics git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7775 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/NetworkPicture.java, htroot/SearchEventPicture.java, htroot/WebStructurePicture_p.java, source/de/anomic/yacy/graphics/NetworkGraph.java |
Sun Jun 05 22:04:41 CEST 2011 by orbiter | added jempbox-1.5.0.jar which is required by pdfbox-1.5 as stated in http://pdfbox.apache.org/dependencies.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7774 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/jempbox-1.5.0.License, lib/jempbox-1.5.0.jar, source/net/yacy/document/parser/pdfParser.java |
Sat Jun 04 20:55:09 CEST 2011 by lotus | equalize lock icon for Status.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7773 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/submenuConfig.template |
Sat Jun 04 15:25:03 CEST 2011 by sixcooler | re-enable POST over proxy, which didn't work since update to httpcore-4.1.1 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7772 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/NonClosingInputStreamEntity.java |
Sat Jun 04 00:34:42 CEST 2011 by suessthomas | Creates a new network definition, yacy.networks.metager.unit. The YaCy freeworld network used in this network definition, minor enhancements for the feed of MetaGer were integrated. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7771 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.metager.unit, defaults/yacy.networks |
Fri Jun 03 01:27:26 CEST 2011 by orbiter | - added more options to access grid picture, web structure picture and network graphics - remove test class git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7770 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessGrid_p.html, htroot/AccessPicture_p.java, htroot/WebStructurePicture_p.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/visualization/GraphPlotter.java |
Wed Jun 01 21:52:05 CEST 2011 by orbiter | added a deploy script that can be used to deploy yacy releases into the current release for testing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7768 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/deploy.sh |
Wed Jun 01 21:49:44 CEST 2011 by orbiter | removed pause in search test git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7767 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/searchtest.sh |
Wed Jun 01 15:27:04 CEST 2011 by f1ori | * improve link rewriting in proxy-url * only rewrites links, which are in current search domain git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7765 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/http/server/HTTPDFileHandler.java |
Tue May 31 18:13:33 CEST 2011 by sixcooler | check filter to bee a correct pattern on edit CrawlProfiles see; http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3277&p=22662#p22660 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7764 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java |
Tue May 31 15:33:04 CEST 2011 by orbiter | added fresh links to java regular expression api-doc git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7763 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartExpert_p.html |
Mon May 30 23:25:20 CEST 2011 by f1ori | * further improve proxyurl git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7762 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Mon May 30 16:56:36 CEST 2011 by orbiter | added latest pdfbox git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7761 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.5.0.jar, lib/fontbox-1.5.License, lib/pdfbox-1.5.0.jar, lib/pdfbox-1.5.License |
Mon May 30 12:16:43 CEST 2011 by orbiter | code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7760 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/WebStructureGraph.java |
Mon May 30 10:53:58 CEST 2011 by orbiter | more concurrency (enhancement) hacks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7759 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/TextSnippet.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/util/FileUtils.java |
Mon May 30 09:51:18 CEST 2011 by orbiter | another resource hack git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7758 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/WebStructureGraph.java |
Mon May 30 09:31:17 CEST 2011 by orbiter | more hacks to omit superfluous string object allocation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7757 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, source/de/anomic/search/MetadataRepository.java |
Mon May 30 06:19:20 CEST 2011 by orbiter | less String object creation during search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7756 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacysearch.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java |
Sun May 29 22:33:12 CEST 2011 by orbiter | lesse memory occupation during ranking and faster host navigator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7755 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/RankingProcess.java |
Fri May 27 18:30:55 CEST 2011 by lotus | show nsis version in installer again for more transparency git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7754 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Fri May 27 16:26:37 CEST 2011 by orbiter | one more hack to free ram for search events git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7753 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEventCache.java |
Fri May 27 14:35:08 CEST 2011 by orbiter | write access/load time to solr index git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7752 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/SolrScheme.java |
Fri May 27 14:34:38 CEST 2011 by orbiter | hack to increase speed of url hash computation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7751 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/util/FileUtils.java |
Fri May 27 14:08:45 CEST 2011 by orbiter | - localsearch test script does also a snippet-fetch - killYACY.sh does not need a sleep between kill -3 and kill -9 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7750 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/localsearch.sh, killYACY.sh |
Fri May 27 13:21:00 CEST 2011 by f1ori | * fix negative result counts results sorted out by add to RankingProcess were counted in sortedout-counter, but were not added to remote_indexCount nor local_indexCount git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7749 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/RankingProcess.java |
Fri May 27 13:18:22 CEST 2011 by orbiter | better memory protection in search logger git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7748 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/AccessTracker.java |
Fri May 27 11:32:03 CEST 2011 by orbiter | - changed snippet fetch strategy logic: do not check if entry is in cache. This should reduce IO load on the HTCACHE which is a showstopper during large number of search requests - forced a possible short memory status when a search is started to flush caches that may cause search-heaps with resource contention effects git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7747 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/kelondro/util/MemoryControl.java |
Thu May 26 18:24:11 CEST 2011 by orbiter | fix for IPv6 localhost proxy client git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7744 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init |
Thu May 26 18:13:07 CEST 2011 by f1ori | * add .gitignore to svn, so it doesn't get lost on git syncronization git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7743 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .gitignore |
Thu May 26 18:09:29 CEST 2011 by f1ori | * fix replacing regexes in url proxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7742 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Thu May 26 16:35:32 CEST 2011 by orbiter | fix for some possible causes of memory leaks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7741 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/SearchEventCache.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/kelondro/blob/Compressor.java |
Thu May 26 14:58:11 CEST 2011 by orbiter | - non-blocking word hash computation with dynamic digest object generation (this was important!) - (very) small performance enhancement in did-you-mean git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7740 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/DidYouMean.java, source/net/yacy/kelondro/order/Digest.java |
Thu May 26 14:35:24 CEST 2011 by orbiter | disabled did-you-mean computation for json and rss search results where this info is not used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7739 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java |
Tue May 24 23:08:01 CEST 2011 by orbiter | - commented out experimental distributed ranking loading - less threads for blocking threads - disable all threads for DHT transmission for networks with zero peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7737 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java |
Tue May 24 12:06:20 CEST 2011 by orbiter | fix for endless loop in FTP crawling, see http://bugs.yacy.net/view.php?id=32 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7736 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java |
Tue May 24 11:31:07 CEST 2011 by orbiter | always use a template method cache even if the template cache flag is set to false. This flag is only used to make dynamic updates to the template files, to not dynamic updates to the rewrite methods (which is not possible without recompiling). low memory usage is guaranteed by the usage of soft references which are dropped before an OOM is thrown git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7735 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java |
Tue May 24 11:14:32 CEST 2011 by orbiter | fix for bug 0000036: no crawling of https pages git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7734 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java |
Sun May 22 22:12:13 CEST 2011 by orbiter | enhanced logging git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7732 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/AccessTracker.java |
Sun May 22 21:34:20 CEST 2011 by orbiter | show dom counter only for active crawls where the dom counter is enabled within the crawl profile git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7731 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.java |
Wed May 18 16:27:24 CEST 2011 by orbiter | these YBR files are not needed any more git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7730 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: |
Tue May 17 17:53:59 CEST 2011 by orbiter | fix in log initialization according to hint from Dominic git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7728 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/Log.java |
Tue May 17 10:57:49 CEST 2011 by orbiter | abstraction of surrogate main element (xmlns:geo was missing for wiki extracts) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7727 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java |
Tue May 17 02:00:01 CEST 2011 by orbiter | removed (almost) deadlocks and unnecessary CPU load git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7726 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/net/yacy/document/parser/html/ContentScraper.java |
Tue May 17 01:19:11 CEST 2011 by orbiter | test file not needed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7725 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: |
Sat May 14 13:30:41 CEST 2011 by orbiter | added timeout setting to scanner interface git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7723 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java |
Sat May 14 11:23:19 CEST 2011 by low012 | *) preventing potential NPE which occured if user deleted DATA/RELEASE manually and opened ConfigureUpdate_p.java then git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7722 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigUpdate_p.java |
Sat May 14 11:13:16 CEST 2011 by orbiter | added bigrange option for network scan git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7721 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Scanner.java |
Sat May 14 11:03:18 CEST 2011 by low012 | *) revert of r7667 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7720 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: stopYACY.sh |
Sat May 14 11:03:16 CEST 2011 by orbiter | speed-up of network port scanner git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7719 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, htroot/Wiki.html, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java |
Sat May 14 10:29:16 CEST 2011 by orbiter | added topmenu as defined for search to wiki git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7718 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Wiki.html, htroot/Wiki.java |
Fri May 13 19:24:03 CEST 2011 by lotus | restart link after memory changed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7717 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Performance_p.html, locales/de.lng |
Fri May 13 17:31:24 CEST 2011 by orbiter | one more fix for SVN 7713 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7716 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Settings_p.java, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/migration.java |
Fri May 13 17:20:57 CEST 2011 by orbiter | fix for svn 7713 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7715 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Settings_p.java, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/migration.java |
Fri May 13 15:40:23 CEST 2011 by apfelmaennchen | transparent progress bar git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7714 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/base.css |
Fri May 13 08:20:00 CEST 2011 by orbiter | show progress of search after display of results is finished git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7712 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacysearch.js, htroot/opensearchdescription.java, htroot/opensearchdescription.xml, htroot/yacysearch.html, htroot/yacysearchlatestinfo.java, htroot/yacysearchlatestinfo.json |
Mon May 09 22:06:05 CEST 2011 by lotus | revert 7710, configure java in firewall for full access. this is the usual config if the user manually accepts java for the firewall. otherwise if a port is specified, windows will not ask on any port change. this would break yacy and other java applications if they run not on the specified port. this would not be an expected behaviour for the user. firewall config may fail for win64 systems (system32 is specified in path) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7711 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Mon May 09 18:20:34 CEST 2011 by Florian Richter | * proxy works almost Changed Files: source/net/yacy/http/CrashProtectionHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/YaCyErrorHandler.java |
Mon May 09 08:16:35 CEST 2011 by pca | Windows Installer: - remove firewall-handling for WinXP (can only open for JRE not for special port) - Vista/Win 7: open port 1900 for communication with router (uPnP) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7710 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Mon May 09 00:47:20 CEST 2011 by orbiter | better handling of on/off switched solr indexing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7709 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/workflow/AbstractBlockingThread.java |
Sun May 08 10:45:58 CEST 2011 by orbiter | fix for very bad compare git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7708 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/Heap.java |
Sat May 07 20:52:20 CEST 2011 by lotus | fix in heuristics config git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7707 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigHeuristics_p.html |
Sat May 07 09:45:02 CEST 2011 by orbiter | another hack to prevent black images git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7706 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewImage.java |
Sat May 07 01:04:27 CEST 2011 by orbiter | fix for wrong search result counter: added a counter for all filtered out entities see also http://bugs.yacy.net/view.php?id=5 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7704 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/RankingProcess.java |
Fri May 06 23:36:35 CEST 2011 by apfelmaennchen | YMarks: sidebar with tabs for tags and folders git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7703 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/test_treeview.html |
Fri May 06 01:11:03 CEST 2011 by orbiter | fix for String Matcher in case that no snippet is returned (NPE) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7702 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/ResultFetcher.java |
Fri May 06 01:05:38 CEST 2011 by orbiter | a fix for the snippet computation and hopefully better snippets git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7701 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/ResultFetcher.java, source/net/yacy/document/SnippetExtractor.java |
Thu May 05 22:03:27 CEST 2011 by lotus | proper jre version (without -s in filename) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7699 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/download.php |
Thu May 05 11:23:10 CEST 2011 by orbiter | removed unused functions in condenser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7698 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/Condenser.java |
Wed May 04 00:21:45 CEST 2011 by orbiter | small fix for solr export git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7696 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/solr/SolrScheme.java |
Tue May 03 23:21:06 CEST 2011 by apfelmaennchen | YMarks: some small changes/fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7695 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_ymark.json, htroot/api/ymarks/import_ymark.java, htroot/yacy/ui/js/jquery-flexigrid.js, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkTables.java |
Mon May 02 22:11:58 CEST 2011 by apfelmaennchen | YMarks: some improvements to flexigrid quick search on YMarks.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7694 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/YMarks.html, htroot/api/ymarks/get_tags.java, htroot/api/ymarks/get_ymark.java, source/de/anomic/data/ymark/YMarkTables.java |
Mon May 02 12:46:05 CEST 2011 by sixcooler | @apfelmaenchen: is this the expected, but forgotten change? Please correct if I'm wrong (this let me build Yacy again) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7692 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java |
Sun May 01 17:49:04 CEST 2011 by orbiter | fix in Document inboundlinks/outboundlinks sorting git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7690 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/Document.java |
Sat Apr 30 21:32:07 CEST 2011 by pca | Windows Installer: - fix for firewall Vista/Win7 - update to JRE 1.6 u25 - TODO: fix for firewall WinXP and setting for uPnP (Port 1900) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7689 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/download.php, build.nsi |
Fri Apr 29 17:36:44 CEST 2011 by orbiter | added more attributes to html evaluation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7688 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java |
Fri Apr 29 04:26:50 CEST 2011 by orbiter | another patch according to http://bugs.yacy.net/view.php?id=26#c36 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7686 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeed.java |
Fri Apr 29 04:19:13 CEST 2011 by orbiter | better logic for proper seed ip recognition and better error messages git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7685 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeed.java |
Fri Apr 29 04:00:37 CEST 2011 by orbiter | better magic git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7684 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/query.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java |
Thu Apr 28 16:08:02 CEST 2011 by orbiter | removed System.out.println git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7682 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverSwitch.java |
Thu Apr 28 15:19:24 CEST 2011 by orbiter | added one more script file to release build script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7681 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Thu Apr 28 15:14:30 CEST 2011 by orbiter | removed unused class and added license header for new class git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7680 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/Evaluation.java |
Thu Apr 28 15:04:33 CEST 2011 by f1ori | * function for simple crawl of one url git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7678 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlStacker.java |
Thu Apr 28 13:18:14 CEST 2011 by f1ori | add ranking weight to json-search-results git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7677 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java, htroot/yacysearchitem.json, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java |
Tue Apr 26 19:39:16 CEST 2011 by sixcooler | not setting userAgent from Constructor as default for following calls git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7676 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Tue Apr 26 13:46:31 CEST 2011 by sixcooler | not using HTTPConnector anymore git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7674 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java |
Thu Apr 21 23:59:56 CEST 2011 by orbiter | added some comments to explain the isLocal patch git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7673 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/Domains.java |
Thu Apr 21 00:54:29 CEST 2011 by low012 | *) fixed typo in stop script *) added <u> </u> tags for underlined text in Wiki Code *) minor code changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7671 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/WikiHelp.html, htroot/env/base.css, source/de/anomic/data/wiki/WikiCode.java, stopYACY.sh |
Thu Apr 21 00:33:01 CEST 2011 by apfelmaennchen | YMark: intermediate work towards flexigrid support git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7670 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/get_ymark.json, htroot/api/ymarks/get_ymark.xml, htroot/yacy/ui/yacyui-bookmarks.html, source/de/anomic/data/ymark/TablesRowComparator.java, source/de/anomic/data/ymark/YMarkTables.java |
Wed Apr 20 19:27:59 CEST 2011 by low012 | *) fixed stupid bug (introduced in r7663 by myself) which caused wrong parsing of Wiki pages git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7669 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java |
Mon Apr 18 21:09:54 CEST 2011 by low012 | *) ./stopYACY:sh -f kills YaCy in case regular shutdown does not work git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7667 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: stopYACY.sh |
Mon Apr 18 00:53:15 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=12 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7665 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.html, htroot/index.java |
Mon Apr 18 00:37:04 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=24 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7664 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/WebStructureGraph.java |
Sun Apr 17 23:02:18 CEST 2011 by low012 | *) removed recursion which hopefully prevents exception *) fixed bug in creation of table of content which caused double entries if a page was previewed more than once git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7663 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java |
Sun Apr 17 00:09:50 CEST 2011 by apfelmaennchen | YMark: more bug fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7662 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/ymark/YMarkHTMLImporter.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkTables.java |
Sat Apr 16 23:19:42 CEST 2011 by apfelmaennchen | YMark: code clean up and some small fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7661 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/test_import.html, source/de/anomic/data/ymark/YMarkHTMLImporter.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkTables.java, source/de/anomic/data/ymark/YMarkXBELImporter.java |
Sat Apr 16 11:09:33 CEST 2011 by apfelmaennchen | YMark: - some improvements to firefox json bookmark importer - test import with: /api/ymarks/test_import.html - view ymarks with: /api/ymarks/test_treeview.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7660 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/import_ymark.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/de/anomic/data/ymark/YMarkTables.java |
Sat Apr 16 05:08:25 CEST 2011 by sixcooler | correct slip of click in classpath from last commit - I wonder there are 7658'is around apflemaenchen, please don't take this amiss git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7659 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist |
Fri Apr 15 02:48:39 CEST 2011 by sixcooler | just cosmetics - keeping my baby clean :-) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7656 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Thu Apr 14 22:12:14 CEST 2011 by orbiter | removed httpclient-3.1 (this was added with last commit which was a mistake) the httpclient is required by solrj but no class from solrj is used which references to httpclient-3.1 Instead the YaCy http client library based on the apache http client 4.1 is used using a wrapper class which is in net.yacy.cora.services.federated.solr.SolrHTTPClient git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7655 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/services/federated/solr/SolrScheme.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java |
Wed Apr 13 17:17:00 CEST 2011 by orbiter | some enhancements to scoring speed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7652 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/storage/ConcurrentScoreMap.java, source/net/yacy/cora/storage/OrderedScoreMap.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/document/WordCache.java |
Wed Apr 13 15:28:28 CEST 2011 by Florian Richter | * caching in proxy Changed Files: source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/YaCySecurityHandler.java |
Wed Apr 13 15:22:27 CEST 2011 by orbiter | fix for mediawiki importer and wikicode parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7651 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java, source/net/yacy/document/importer/MediawikiImporter.java |
Wed Apr 13 08:16:31 CEST 2011 by orbiter | added new SearchResult class (to be used later) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7650 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchResult.java |
Sat Apr 09 11:48:44 CEST 2011 by Florian Richter | * implement proxy Changed Files: lib/jetty-servlet-7.3.0.v20110203.jar, lib/jetty-servlets-7.3.0.v20110203.jar, source/net/yacy/http/HttpServer.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/YacyDomainHandler.java |
Thu Apr 07 09:31:17 CEST 2011 by f1ori | * fix permissions of bin/-folder in debian package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7647 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/yacyInit.m4, build.xml |
Wed Apr 06 12:59:31 CEST 2011 by cominch | different initialization of empty variables in alternative constructor. This leads to wrong interpretation of user credentials, resulting in unnecessary "@" in front of host, and different urlhash values. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7646 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Tue Apr 05 10:04:25 CEST 2011 by orbiter | better logging for robinson selection according to peer tag git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7645 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacySeed.java |
Tue Apr 05 00:01:07 CEST 2011 by orbiter | removed the DNS resolving for web structure computation from the indexing queue and placed it in a concurrent computation queue that does not block the crawler. Makes crawling faster and less DNS-speed-dependent git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7644 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/graphics/WebStructureGraph.java |
Mon Apr 04 22:31:02 CEST 2011 by orbiter | reverted synchronization from SVN 7641 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7643 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Mon Apr 04 12:13:30 CEST 2011 by orbiter | removed synchronization again... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7641 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Mon Apr 04 11:47:18 CEST 2011 by orbiter | fixes to crawler and new user-agent crawl-delay handling git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7640 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/robotsParser.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/repository/LoaderDispatcher.java |
Sat Apr 02 12:58:56 CEST 2011 by f1ori | * add bin-folder to debian package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7638 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Sat Apr 02 11:25:07 CEST 2011 by low012 | *) preventing whitespace keys in config file git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7637 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigProperties_p.java |
Sat Apr 02 02:11:37 CEST 2011 by orbiter | - enhancements in surrogate reading - better display of map in location search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7636 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch_location.html, htroot/yacysearch_location.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/services/SearchSRURSS.java |
Fri Apr 01 13:05:42 CEST 2011 by orbiter | enhanced the surrogate parser: better reading of UTF-8 characters git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7634 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/document/content/SurrogateReader.java |
Thu Mar 31 11:41:30 CEST 2011 by orbiter | - added an index constraint 'has location' to the condenser - added evaluation of the 'has location' constraint to search using the /location operator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7633 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/yacysearch.java, source/de/anomic/search/RankingProcess.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/Condenser.java |
Thu Mar 31 11:40:01 CEST 2011 by orbiter | fix for url normalization (no backpath resolving in post parameters) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7632 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Wed Mar 30 10:17:25 CEST 2011 by orbiter | fix for http://bugs.yacy.net/view.php?id=8 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7630 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/io/BufferedRecords.java |
Mon Mar 28 21:55:15 CEST 2011 by f1ori | * show info in log, if robots.txt is rejected due to wrong mime-type git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7628 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/RobotsTxt.java |
Sat Mar 26 16:11:29 CET 2011 by lotus | write PID to yacy.running git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7627 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/yacy.java |
Sat Mar 26 00:18:48 CET 2011 by Florian Richter | * yacy domain handler for jetty * rewrite from / to /index.html Changed Files: .classpath, build.xml, source/de/anomic/http/server/AlternativeDomainNames.java, source/net/yacy/http/ContentModHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/RewriteHandler.java, source/net/yacy/http/TemplateHandler.java, source/net/yacy/http/YacyDomainHandler.java |
Thu Mar 24 07:24:37 CET 2011 by lotus | hups git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7626 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/ui/js/jquery.autocomplete.yacy.js |
Wed Mar 23 21:22:08 CET 2011 by lotus | better rename modified version git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7625 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/index.html, htroot/yacy/ui/js/jquery.autocomplete.yacy.js, htroot/yacyinteractive.html, htroot/yacysearch.html |
Wed Mar 23 21:13:43 CET 2011 by lotus | do not autocomplete on mouseover. this has resulted in unwanted autocomplete. fixes bug #3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7624 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacy/ui/js/jquery.autocomplete.js |
Wed Mar 23 01:48:19 CET 2011 by low012 | *) fixed Exceptions which caused 500 error when entering invalid URL mask or invalid prefer mask, invalid masks are ignored, error message is displayed on yacysearch.html (what about yacysearch.rss and yacysearch.json?) *) fixed "more options" link on yacysearch.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7623 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/search/QueryParams.java |
Wed Mar 23 01:44:38 CET 2011 by low012 | *) fixed NPE which occured when empty String was entered as search word git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7622 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEvent.java |
Tue Mar 22 12:00:55 CET 2011 by orbiter | better concurrency in ViewImage icon cache and OOM protection for too large icon caches git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7621 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewImage.java |
Tue Mar 22 10:34:10 CET 2011 by orbiter | - enhancements to DNS IP caching and crawler speed - bugfixes (NPEs) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7619 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexImportWikimedia_p.java, htroot/PerformanceMemory_p.java, source/de/anomic/crawler/Balancer.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/importer/MediawikiImporter.java |
Mon Mar 21 23:23:20 CET 2011 by low012 | *) <mode="Homer">D'oh!</Homer> git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7618 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java |
Mon Mar 21 23:20:19 CET 2011 by low012 | *) trying to fix ConcurrentModificationException git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7617 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java |
Mon Mar 21 20:42:22 CET 2011 by f1ori | * fix bug #7 * log requires poison to finish, so Base64Order main-function doesn't finish, when called from debian configure script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7616 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/yacyInit.m4, debian/control, source/net/yacy/kelondro/order/Base64Order.java |
Mon Mar 21 19:02:09 CET 2011 by low012 | *) fixed potential "java.lang.IllegalArgumentException: Illegal group reference" which occured if special characters which are also used as metacharacters in regular expression were used inside of <pre>...</pre> (see: http://veerasundar.com/blog/2010/01/java-lang-illegalargumentexception-illegal-group-reference-in-string-replaceall/) The class still contains a potential ConcurrentModificationException which occurs when the List which contains the elements of the table of content is moified during a recursion of tagReplace(). Will try to fix this later today. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7615 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/WikiCode.java |
Sun Mar 20 18:59:58 CET 2011 by f1ori | * fix bug #6 * exclude signature files from auto-deletion of unknown files in DATA/RELEASE git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7612 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigUpdate_p.java, source/de/anomic/yacy/yacyRelease.java |
Sun Mar 20 09:36:29 CET 2011 by orbiter | added a clearall.sh script that deletes the complete index and everything else that belongs to crawling git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7611 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: bin/clearall.sh, build.xml |
Sun Mar 20 00:58:11 CET 2011 by orbiter | added script for importmediawiki.sh in build.xml git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7610 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Thu Mar 17 22:09:18 CET 2011 by orbiter | performance hack for Balancer and ip address parsing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7608 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/Balancer.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Thu Mar 17 22:07:44 CET 2011 by orbiter | fix/enhancement in Crawler: do not generate domain match pattern if crawl depth is 0 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7607 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java |
Thu Mar 17 21:48:39 CET 2011 by orbiter | another anhancement to the ThreadDump function: better multiple dumps and filtering out of not interesting dump parts git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7606 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Threaddump_p.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Thu Mar 17 20:40:05 CET 2011 by Florian Richter | * authentication complete (using old credentials from config file) Changed Files: source/net/yacy/http/YaCyLegacyCredential.java, source/net/yacy/http/YaCyLoginService.java, source/net/yacy/http/YaCySecurityHandler.java |
Thu Mar 17 18:09:19 CET 2011 by orbiter | fix for not-deleted .gap and .idx files see also: http://forum.yacy-websuche.de/viewtopic.php?p=22128#p22128 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7605 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java |
Wed Mar 16 17:39:31 CET 2011 by Florian Richter | * authentication implemented with own securityhandler Changed Files: source/net/yacy/http/HttpServer.java, source/net/yacy/http/TemplateHandler.java, source/net/yacy/http/YaCyLoginService.java, source/net/yacy/http/YaCySecurityHandler.java |
Wed Mar 16 08:05:10 CET 2011 by orbiter | fix for active dns resolving: do not resolve in case that the dns server is not available (offline mode) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7604 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Wed Mar 16 00:05:51 CET 2011 by orbiter | fix for multiple thread dump git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7603 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Tue Mar 15 10:52:39 CET 2011 by orbiter | avoided using a synchronized(this) for the hash computation to prevent that the lock on the object is (accidently) stolen by another thread and replaced this synchronization using the protocol object. Made also the protocol object final. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7602 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/kelondro/data/meta/DigestURI.java |
Tue Mar 15 09:39:05 CET 2011 by orbiter | fix for bug in formatting in ThreadDump and added hint for linux/Mac users that they may use the LOCKED feature using the start option -l git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7601 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Threaddump_p.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Tue Mar 15 01:47:30 CET 2011 by orbiter | avoid synchronization in DigestURI hash requests git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7599 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/DigestURI.java |
Mon Mar 14 23:52:30 CET 2011 by orbiter | removed more warnings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7598 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/storage/OrderedScoreMap.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/util/FileUtils.java |
Mon Mar 14 22:34:42 CET 2011 by orbiter | removed debug line git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7597 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 14 22:33:36 CET 2011 by orbiter | patch for Windows: do not use the thread lock feature from previous commit if used on Windows git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7596 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 14 22:32:20 CET 2011 by orbiter | added new feature for Thread dump: "THREADS WITH STATES: LOCK FOR OTHERS" will show only such threads that lock other threads. This is the 'opposite part' of the blocked threads. Because that this uses a thread dump that is produced with a kill -3 on the PID of the process and such thread dumps are written by the Java core outside of System.out and Sytem.err it is necessary to read the dump from a log in the file system. Such a log is only written if YaCy is started with startYACY.sh on a linux system. That means: this feature is only available on linux and Mac OS X if YaCy is started with ./startYACY.sh -l git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7595 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Threaddump_p.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 14 22:08:18 CET 2011 by orbiter | removed type cast warnings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7594 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/storage/OrderedScoreMap.java |
Mon Mar 14 22:07:15 CET 2011 by orbiter | removed type cast warnings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7593 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/order/CloneableMapIterator.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/util/FileUtils.java |
Mon Mar 14 22:03:37 CET 2011 by orbiter | removed unused variables git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7592 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/visualization/GridTree.java |
Mon Mar 14 21:48:36 CET 2011 by orbiter | disabled some of the previous changes (did not work in openjdk) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7591 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 14 21:46:04 CET 2011 by orbiter | update to thread dump logs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7590 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Threaddump_p.java, source/net/yacy/kelondro/logging/ThreadDump.java, source/net/yacy/kelondro/util/OS.java |
Mon Mar 14 21:17:01 CET 2011 by Florian Richter | * SSI work with jetty, it's pretty usable now Changed Files: source/net/yacy/http/ContentModHandler.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/SSIHandler.java, source/net/yacy/http/TemplateHandler.java |
Mon Mar 14 20:32:08 CET 2011 by pca | Windows Installer: - add support for Windows Firewall on Win XP (SP2/SP3), Vista and Win 7 (open port 8090) - this should cover almost every Windows installation at home git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7589 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Sun Mar 13 23:44:50 CET 2011 by sixcooler | also putting jcifs-1.3.15 into classpath (let me me build YaCy again :-) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7588 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: .classpath |
Sun Mar 13 21:09:52 CET 2011 by suessthomas | Update jcifs-library to Version 1.3.15. Small Changes, read: http://jcifs.samba.org/ - "Minor adjustments have been applied to DcerpcHandle locking routines in the SID class to fix sporadic occurances of "All pipe instances are busy" errors under high load." git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7587 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/jcifs-1.3.15-LICENSE.txt, lib/jcifs-1.3.15.jar |
Sat Mar 12 23:54:41 CET 2011 by orbiter | more enhancements against blocked threads occurred in seed age evaluation (blocks httpd in some cases) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7585 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java |
Fri Mar 11 13:15:56 CET 2011 by Florian Richter | * jetty works for many pages now Changed Files: build.xml, htroot/Status.java, source/net/yacy/http/HttpServer.java, source/net/yacy/http/TemplateHandler.java, source/net/yacy/yacy.java |
Thu Mar 10 21:04:15 CET 2011 by lotus | fix basic config change in portal mode git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7582 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.java |
Thu Mar 10 20:30:05 CET 2011 by pca | Windows Installer: - check Windows-Version on startup, support only Windows 2000 and newer (necessary for Sun-JRE and as preparation for firewall section) - little changes in JRE section handling git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7581 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Wed Mar 09 21:36:01 CET 2011 by lotus | adding extension for parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7579 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/htmlParser.java |
Wed Mar 09 21:19:34 CET 2011 by pca | Windows Installer: - detect JRE at startup, showing install-option depends on result - hide window for external call "attrib" - some cleanup and restructure for readability git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7578 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Wed Mar 09 21:08:20 CET 2011 by orbiter | fix for broken JSON, see: http://forum.yacy-websuche.de/viewtopic.php?p=22162#p22162 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7577 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java |
Wed Mar 09 18:04:34 CET 2011 by orbiter | fix for 'invisible entry', see http://forum.yacy-websuche.de/viewtopic.php?p=22133#p22133 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7576 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/CrawlQueues.java |
Wed Mar 09 10:53:57 CET 2011 by orbiter | added default size to StringBuffer in logger (and it is not possible to replace the StringBuffer with a StringBuilder...) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7573 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/SimpleLogFormatter.java |
Wed Mar 09 00:27:41 CET 2011 by low012 | *) minor changes *) fixed potential NPE in suggest.java git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7571 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/WebStructurePicture_p.java, htroot/suggest.java, htroot/yacyinteractive.java, htroot/yacysearch.java |
Tue Mar 08 23:37:17 CET 2011 by low012 | *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7570 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewLog_p.java, htroot/yacysearch.java, htroot/yacysearchitem.java |
Tue Mar 08 20:11:49 CET 2011 by pca | Cleanup for last commit (forget some unnecessary parts) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7569 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Tue Mar 08 20:07:22 CET 2011 by pca | Add check for hard drive and free space to Windows-Installer git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7568 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Tue Mar 08 00:32:01 CET 2011 by orbiter | better performance computing search targets with index abstracts git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7566 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEvent.java |
Tue Mar 08 00:12:39 CET 2011 by orbiter | enhanced secondary search: index abstracts decompression is now much faster and does not cause strong CPU load after several searches with more than one word git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7565 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/SetTools.java |
Tue Mar 08 00:05:23 CET 2011 by low012 | *) set SVN properties git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7564 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Mon Mar 07 22:44:58 CET 2011 by low012 | *) added the only changes from r7557 which actualy made sense *) caught potential exception (occured when user entered a string which did not contain digits only for the maximum number of lines) *) use prop.putHTML to avoid potential XSS attack in case an attacker manages to cause something to end up in the logs which contains a string which was defined by the attacker git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7562 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewLog_p.java |
Mon Mar 07 22:24:46 CET 2011 by orbiter | use less peers for remote search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7561 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/dht/PeerSelection.java |
Mon Mar 07 22:19:47 CET 2011 by orbiter | reverted also changes in ViewLog from SVN 7557 because the ThreadDump submenu was not visible any more. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7560 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewLog_p.html, htroot/ViewLog_p.java |
Mon Mar 07 19:55:11 CET 2011 by low012 | *) removed unused code (I assume that most of the code was really dead, but if you need any of the classes, tell me and I will put it back in.) *) minor code cleanup in ViewLog git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7557 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewLog_p.html, htroot/ViewLog_p.java |
Mon Mar 07 16:29:45 CET 2011 by orbiter | less synchronization, better thread dump tool git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7556 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Threaddump_p.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Sun Mar 06 13:30:22 CET 2011 by orbiter | - even less memory for circle tool - background thread for bookmark initialization: this uses a DNS lookup which may cause long waiting times during startup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7554 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/visualization/CircleTool.java |
Sun Mar 06 12:23:04 CET 2011 by orbiter | reduce size of static memory usage: use short instead of int in circle coordinates cache git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7553 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/visualization/CircleTool.java |
Sun Mar 06 02:01:29 CET 2011 by orbiter | removed (most probably!?!) superfluos System.err output git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7552 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/logging/Log.java |
Sat Mar 05 20:52:34 CET 2011 by orbiter | one more patch again OOM during secondary remote search see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3202 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7551 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/util/ByteBuffer.java |
Fri Mar 04 15:34:37 CET 2011 by orbiter | this asserts prevents a network operation in case of sabotage and must be removed therefore git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7550 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacySeedDB.java |
Fri Mar 04 15:29:22 CET 2011 by orbiter | small bug fixes and experiments with search speed enhancement git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7549 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/NetworkPicture.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/SplitTable.java |
Fri Mar 04 11:00:37 CET 2011 by orbiter | fix for bug in deletion of old seeds git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7547 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java |
Thu Mar 03 21:44:53 CET 2011 by lotus | jre update git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7546 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: addon/download.php |
Thu Mar 03 19:34:32 CET 2011 by lotus | remove old libraries in case of update git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7545 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Thu Mar 03 12:30:04 CET 2011 by orbiter | memory/performance hacks, less synchronization, better concurrency git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7544 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/util/FileUtils.java |
Thu Mar 03 05:16:33 CET 2011 by sixcooler | stupid me: revert placement of closing connection which caused unclosed connections + reuse sockets git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7543 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Tue Mar 01 14:50:11 CET 2011 by sixcooler | Not storing external header in an Header-Array and reduce a loop for its conversion. Ensure connection close if a OOM is thrown. Ensure setting resolved host is set at the request. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7542 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Mon Feb 28 07:28:29 CET 2011 by orbiter | better concurrency (less locking on date formatting) more logging and minor bug fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7540 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/util/MapTools.java |
Sun Feb 27 22:39:38 CET 2011 by f1ori | * implement url proxy, a proxy via the url http://peer:port/proxy.html?url=http://domain.tld/path * enable with proxyURL = true * could be useful to browse specific pages with proxy or use own improvements in proxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7538 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/http/server/HTTPDFileHandler.java |
Sun Feb 27 16:01:46 CET 2011 by mikeworks | Tray.java: Broke the build on with wrong non UTF-8 encoded file and french umlauts (unmappable character for encoding UTF8) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7537 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/Tray.java |
Sun Feb 27 12:42:33 CET 2011 by mikeworks | Tray.java: Added localization for french tray icon command - although this can probably also done better than with if statements. (preferably also from the locales file) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7536 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/Tray.java |
Sun Feb 27 02:19:46 CET 2011 by mikeworks | de.lng: Updated German translation added missing line no change when no other network was selected YaCy logos: removed outer white line that made the logo blitz on black or dark themes http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1724#p21655 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7535 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/grafics/YaCyLogo_120ppi.png, htroot/env/grafics/YaCyLogo_60ppi.png, htroot/env/grafics/yacy.gif, locales/de.lng |
Sat Feb 26 10:27:15 CET 2011 by orbiter | more logging for OOMs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7534 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/cora/protocol/TimeoutRequest.java |
Fri Feb 25 22:29:55 CET 2011 by orbiter | performance hack: excluded map encoding in many cases from synchronization block, especially when doing an iteration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7532 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java |
Fri Feb 25 14:26:09 CET 2011 by orbiter | added convenience class to generate UTF StringBody objects with a default UTF8 charset. Reason: if this is not used in StringBody-Class initialization, a default charset name is parsed. This is a synchronized process and all classes using default charsets synchronize at that point Synchronization is omitted if this class is used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7530 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/net/yacy/cora/document/UTF8.java, source/net/yacy/cora/services/SearchSRURSS.java |
Fri Feb 25 12:58:01 CET 2011 by orbiter | speed hacks in BLOB ArrayStack: - more concurrency if possible - less threads if no concurrency necessary git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7527 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java |
Fri Feb 25 12:23:57 CET 2011 by orbiter | performance enhancements using an alternative to a insensitive collator (a complex string compare): - less synchronizations - better speed ..at most important and commonly used classes: http headers, url parsing and html parsing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7526 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessPicture_p.java, source/net/yacy/cora/document/ASCIIComparator.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/visualization/HexGridPlotter.java |
Fri Feb 11 16:49:46 CET 2011 by Florian Richter | * augmented proxy Changed Files: source/de/anomic/http/server/HTTPDProxyHandler.java, source/net/yacy/cider/AugmentedHtmlStream.java |
Fri Feb 25 02:19:39 CET 2011 by orbiter | enhancement in synchronisation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7525 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/yacySeed.java |
Fri Feb 25 02:16:15 CET 2011 by sixcooler | next try to fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 tested proxy, crawl, updatedownload - please do further testing! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7524 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Fri Feb 25 01:06:31 CET 2011 by orbiter | more explanation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7522 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Fri Feb 25 01:02:04 CET 2011 by orbiter | emergency commit (sorry sixcooler for not waiting) because without that automatic updating peers would not be able to do the next update. Please see http://forum.yacy-websuche.de/viewtopic.php?p=22059#p22059 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7521 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Fri Feb 25 00:15:27 CET 2011 by sixcooler | fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7520 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Thu Feb 24 11:32:46 CET 2011 by orbiter | - fixed bug in ordering - fixed ConcurrentModificationException in set join git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7519 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Segments.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/util/SetTools.java |
Thu Feb 24 00:04:05 CET 2011 by orbiter | - fix for missing favicons in search widgets - fix for bad digest/hash computation in case of interrupts to class git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7518 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java, source/de/anomic/tools/cryptbig.java, source/net/yacy/kelondro/order/Digest.java |
Wed Feb 23 16:02:00 CET 2011 by lotus | fix missing translation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7517 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: locales/de.lng |
Wed Feb 23 12:28:12 CET 2011 by orbiter | - do not delete idx and gap files if the heap is not modified this change may have bugs in it which may cause damage to your existing data. please use with care. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7516 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/util/FileUtils.java |
Wed Feb 23 10:42:01 CET 2011 by orbiter | - increased default peer ping time from 2 minutes to 1 minute - filtering out too old peers when reading seed lists (limit is now 240 minutes) - added concurrent host names resolving in front of the http client because the http client uses the java built-in DNS resolve which is not multithreading-safe (i have seen deadlocks in thread dumps showing that this bug in jdk is still there) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7515 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java |
Tue Feb 22 23:58:19 CET 2011 by orbiter | added dns resolve to HTTPClient POST using a dns cache to prevent that that not-thread-safe built-in dns cache inside apache http client is used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7513 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/kelondro/data/meta/DigestURI.java, test/net/yacy/cora/document/MultiProtocolURITest.java |
Tue Feb 22 14:33:28 CET 2011 by orbiter | removed unnecessary recording function that caused also a performance problem after serving too much files git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7512 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/http/server/HTTPDFileHandler.java |
Tue Feb 22 13:56:25 CET 2011 by orbiter | - removed synchronization in serverSwitch which should improve speed - fixed wrong assert in network graph - enhanced double check method in table class git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7511 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/FileUtils.java |
Tue Feb 22 11:57:38 CET 2011 by orbiter | initialization of libraries concurrently (faster start-up) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7510 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java |
Mon Feb 21 16:24:09 CET 2011 by orbiter | slightly less restrictive values for DoS git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7509 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java |
Mon Feb 21 16:11:03 CET 2011 by orbiter | DoS to prevent online snippet fetch: allow read from cache. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7508 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java |
Mon Feb 21 13:11:12 CET 2011 by orbiter | added link to svn log git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7507 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/header.template |
Mon Feb 21 12:53:45 CET 2011 by orbiter | - removed usage of /etc/alternatives/www-browser because of problems with lynx, see: http://forum.yacy-websuche.de/viewtopic.php?p=21959#p21959 please look if the browser that is linked with /etc/alternatives/www-browser can be detected and insert call again if it can be made sure that this does not call lynx - replaced severe warnings with just warnings in yacyClient git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7506 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyClient.java, source/net/yacy/gui/framework/Browser.java |
Mon Feb 21 11:51:15 CET 2011 by orbiter | fix in ftp client git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7505 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java |
Mon Feb 21 11:30:58 CET 2011 by orbiter | - disabled arrow keys in search (pages switch) again -- too disturbing when editing the query line - fixed alt text in AccessGrid git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7504 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessGrid_p.html, htroot/yacysearch.html |
Mon Feb 21 10:51:05 CET 2011 by orbiter | more logging, more care about OOMs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7503 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/parser/html/ContentScraper.java |
Mon Feb 21 01:38:49 CET 2011 by orbiter | healing for some OOM problems git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7502 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverCore.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java |
Mon Feb 21 01:07:00 CET 2011 by orbiter | added log output to find bug in url parser for short hosts git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7501 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Mon Feb 21 01:03:11 CET 2011 by orbiter | - enhanced circle drawing speed - beautified 'moving dot' feature (using smaller and correctly positioned dots) - added moving dots to DHT transfer git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7500 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessPicture_p.java, htroot/imagetest.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/GridTree.java, source/net/yacy/visualization/HexGridPlotter.java, source/net/yacy/visualization/RasterPlotter.java |
Sat Feb 19 23:02:44 CET 2011 by suessthomas | Small correction git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7499 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: locales/de.lng |
Sat Feb 19 00:25:58 CET 2011 by orbiter | added missing file for latest commit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7497 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/visualization/HexGridPlotter.java |
Sat Feb 19 00:19:35 CET 2011 by orbiter | THE GRID is coming to YaCy .. see new animated graphics on http://localhost:8090/AccessGrid_p.html showing incoming and outgoing connections in an animated way git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7496 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessGrid_p.html, htroot/AccessGrid_p.java, htroot/AccessPicture_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuAccessTracker.template, source/de/anomic/yacy/graphics/NetworkGraph.java |
Fri Feb 18 03:11:19 CET 2011 by low012 | *) removed link to LogStatistics *) removed translation of LogStatistics Once again: If you want this page back, I will put it back in, but it seems to have been non-operational for over one year without anybody complaining about it. Does anybody know if net.yacy.kelondro.logging.LogalizerHandler still serves any purpose? git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7495 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/submenuViewLog.template, locales/de.lng |
Fri Feb 18 02:25:46 CET 2011 by low012 | *) fixed bug which caused entries to not be deleted when deleting by URL on IndexCreateWWWLocalQueue_p.html (I hope this did not break anything else) *) cleaned up code a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7493 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/api/queues_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/NoticedURL.java |
Fri Feb 18 00:36:46 CET 2011 by orbiter | maybe terminateOldSessions is too slow, removed sleep git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7492 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverCore.java |
Thu Feb 17 23:47:39 CET 2011 by orbiter | removed unused properties git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7491 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.json |
Thu Feb 17 21:42:35 CET 2011 by low012 | *) Oops! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7490 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/Crawler.js |
Thu Feb 17 21:40:34 CET 2011 by low012 | *) make sure that only positive values are accepted as refresh interval on Crawler Monitor page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7489 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/Crawler.js |
Thu Feb 17 17:47:05 CET 2011 by orbiter | fix for non-valid json, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3179 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7488 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchtrailer.java |
Thu Feb 17 02:22:22 CET 2011 by orbiter | moved the main update url - because of the many languages we support now on yacy.net git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7487 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.network.freeworld.unit |
Wed Feb 16 15:53:47 CET 2011 by orbiter | update to browser start with linux git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7486 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/gui/framework/Browser.java |
Wed Feb 16 14:51:30 CET 2011 by orbiter | replaced utf-8 with UTF-8 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7485 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexImportWikimedia_p.html, htroot/api/ymarks/info.html, htroot/processing/domaingraph/applet/index.html, htroot/yacy/ui/yacyui-widget.html, htroot/yacy/ui/yacyuisearch.html |
Tue Feb 15 15:26:25 CET 2011 by orbiter | fix for (false) oom git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7484 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/index/RowCollection.java |
Tue Feb 15 00:16:47 CET 2011 by orbiter | added patch from quix0r, see http://forum.yacy-websuche.de/viewtopic.php?p=21188#p21188 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7483 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_API_p.html, htroot/Table_API_p.java |
Mon Feb 14 23:17:59 CET 2011 by orbiter | more changes to headless mode; now non-headless mode is used when: - YaCy runs on Windows - YaCy is started with the -gui option in all other cases YaCy runs in headless mode git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7481 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/yacy.java |
Mon Feb 14 17:15:14 CET 2011 by orbiter | new methods to open a browser - if YaCy is started with the option -gui, it is not in headless mode. Then the java 1.6 browse method is used if all other methods fail - in linux, the path /etc/alternatives/www-browser is used if no firefox is installed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7480 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/yacy.java |
Sat Feb 12 00:37:13 CET 2011 by low012 | *) replaced all IPs in IP filters for proxy with the proper regular expression *) some cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7477 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, defaults/yacy.network.allip.unit, defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.webportal.unit, source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/cora/protocol/Domains.java |
Thu Feb 10 10:08:44 CET 2011 by lotus | fix: same number format display before and after get/post git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7476 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlURLs_p.java |
Wed Feb 09 22:32:45 CET 2011 by lotus | fix display of statistics link on network.html (caused by menu option) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7475 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.java |
Tue Feb 08 22:53:20 CET 2011 by sixcooler | update to httpclient-4.1 - sorry forgot some git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7474 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/http/GzipDecompressingEntity.java, source/net/yacy/cora/protocol/http/HTTPClient.java |
Tue Feb 08 10:12:48 CET 2011 by f1ori | * add ant-target deleteData, to simulate fresh yacy installation * useful in IDEs like eclipse git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7472 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Sun Feb 06 19:07:08 CET 2011 by low012 | *) fixed bug which affected blacklist entries which consisted of domain _and_ path parts *) minor cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7471 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/BlacklistFile.java, source/net/yacy/repository/FilterEngine.java |
Sun Feb 06 17:00:45 CET 2011 by low012 | *) more input from Moubai git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7470 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: locales/fr.lng |
Sat Feb 05 12:20:25 CET 2011 by low012 | *) updated French langugage file with input from Moubai (http://yacy-forum.org/viewtopic.php?p=1646) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7469 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: locales/fr.lng |
Fri Feb 04 14:25:28 CET 2011 by orbiter | increased number of target peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7468 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/SearchEvent.java |
Thu Feb 03 11:43:09 CET 2011 by orbiter | fix for http://forum.yacy-websuche.de/viewtopic.php?p=21763#p21763 and another fix for non-working global search when search options are switched off git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7467 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigPortal.java, htroot/index.html, htroot/index.java, source/de/anomic/http/server/HTTPDFileHandler.java |
Wed Feb 02 16:28:41 CET 2011 by f1ori | * fix patchCharsetEncoding-test (patchCharsetEncoding now returns null on input null) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7465 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: test/de/anomic/document/parser/htmlParserTest.java |
Wed Feb 02 02:19:25 CET 2011 by orbiter | patch for bad data in url metadata git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7464 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataRow.java |
Wed Feb 02 01:50:06 CET 2011 by orbiter | added custom targets for media search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7463 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.html, htroot/yacysearchitem.java |
Wed Feb 02 01:22:00 CET 2011 by orbiter | replaced more double types with float git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7462 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/RankingProcess.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/storage/ScoreCluster.java |
Mon Jan 31 20:55:00 CET 2011 by orbiter | added link on ViewProfile.html to EditProfile_p.html to make it easier to customize the peer the link appears only if the user has admin rights git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7459 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ViewProfile.html, htroot/ViewProfile.java |
Mon Jan 31 15:25:09 CET 2011 by f1ori | * fix StringIndexOutOfBoundException in WebStructureGraph * add better escaping to saveMap and loadMap git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7458 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/net/yacy/kelondro/util/FileUtils.java |
Fri Jan 28 18:26:47 CET 2011 by low012 | *) less HTML errors, but still several left git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7457 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlProfileEditor_p.html, htroot/Table_API_p.html |
Fri Jan 28 17:24:33 CET 2011 by low012 | *) added a Set to which filter elements are written before mustmatch-filter is created to avoid huge lists of double elements in mustmatch-filter when starting a crawl from a "Link-List of URL" on CrawlStartSite_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7456 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java |
Fri Jan 28 12:32:31 CET 2011 by orbiter | better servlet naming in index administration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7455 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/env/templates/submenuIndexControl.template |
Fri Jan 28 05:46:42 CET 2011 by mikeworks | de.lng: Updated German translation and cleaned removed lines - no unused strings log entries git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7452 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: locales/de.lng |
Thu Jan 27 12:17:11 CET 2011 by orbiter | - added recording of all accessed and submitted servlets - this recording is then used to redirect from the Status.html page to BasicConfig in case that servlet was never submitted - this acts as an addition to the new default pop-up page 'index.html' which offers an administration link to Status.html. For a first-time user this then redirects directly to the former start page BasicConfig.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7451 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/Status.html, htroot/Status.java, htroot/env/templates/header.template, source/de/anomic/http/server/HTTPDFileHandler.java |
Thu Jan 27 11:16:14 CET 2011 by orbiter | - fixed/enhanced move to SD/16:9 images (network, web structure) - added logging in peer ping to analyse time-consuming elements which could be cause for disappearing peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7450 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.java, htroot/NetworkPicture.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/yacy/hello.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java |
Wed Jan 26 15:58:28 CET 2011 by orbiter | - search page becomes default start page (new users are not forced to do configuration since this is not necessary) - adjusted top menu on search page (shows less stuff and now also the network graphics) - adjusted the network page (looks better in when showing no other navigation on top) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7448 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/Network.html, htroot/env/templates/simpleheader.template |
Wed Jan 26 14:59:55 CET 2011 by orbiter | option in network servlet to show the servlet with simple or embed menus git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7447 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.html, htroot/Network.java |
Wed Jan 26 14:52:46 CET 2011 by orbiter | enhanced network graphic git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7446 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.html, htroot/NetworkPicture.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/visualization/PrintTool.java |
Sun Jan 23 19:36:44 CET 2011 by mikeworks | de.lng: Added German translation to Network Scanner menu CrawlStartScanner_p.html: Modified HTML code to validate HTML 4.01 Strict, switched selectAll Javasript, added end tags to input, etc ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7445 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, locales/de.lng |
Sat Jan 22 11:53:36 CET 2011 by orbiter | better table layout in Network and API Administration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7444 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.html, htroot/Table_API_p.html, htroot/env/base.css |
Sat Jan 22 10:52:36 CET 2011 by orbiter | added more php mime types git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7443 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/httpd.mime |
Sat Jan 22 10:50:36 CET 2011 by orbiter | removed yacy.network.group - this feature was never used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7442 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/search/Switchboard.java |
Sat Jan 22 03:11:18 CET 2011 by low012 | *) added lock symbol to "API Action Steering" link in menu since the page is protected git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7440 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/env/templates/header.template |
Fri Jan 21 19:09:24 CET 2011 by lotus | fix bug in installer in case there was a 32 bit jre installed on a 64 bit windows. now the necessary 64 bit jre gets installed. danke PCA42! http://forum.yacy-websuche.de/viewtopic.php?p=21631#p21631 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7439 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.nsi |
Wed Jan 19 10:31:56 CET 2011 by orbiter | more logging for strange network loading bug git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7438 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverSwitch.java |
Mon Jan 17 16:04:00 CET 2011 by f1ori | * create cleanupjob for cached failed urls git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7437 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, source/de/anomic/data/WorkTables.java, source/de/anomic/search/Switchboard.java |
Sat Jan 15 19:49:51 CET 2011 by lotus | adding convenient option for contacting peer on network page 4. can be used on creating new network. this page is hidden from the interface (not linked). git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7435 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Network.html, htroot/Network.java |
Sat Jan 15 14:16:40 CET 2011 by hermens | Let SHORT_MILSEC_FORMATTER make a new formatted String every millisecond see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3103 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7434 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/date/GenericFormatter.java |
Wed Jan 12 13:04:18 CET 2011 by orbiter | catch a OOM in HeapReader iteration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7433 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/HeapReader.java |
Wed Jan 12 01:28:08 CET 2011 by low012 | *) General private License? ;-) *) minor code changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7432 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/net/yacy/cora/services/SearchHub.java |
Mon Jan 10 22:40:56 CET 2011 by f1ori | * provide option to delete cached snippet fetching failures git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7429 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigHTCache_p.html, htroot/ConfigHTCache_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java |
Mon Jan 10 18:47:23 CET 2011 by low012 | *) Fixed bug where a number followed by characters was allowed as a valid value for the amount of memory which YaCy is allowed to consume. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7427 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: reconfigureYACY.sh |
Mon Jan 10 09:40:41 CET 2011 by orbiter | refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7426 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch_location.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/services/SearchAccumulator.java, source/net/yacy/cora/services/SearchHub.java, source/net/yacy/cora/services/SearchSRURSS.java, source/net/yacy/document/SnippetExtractor.java |
Wed Jan 05 18:35:36 CET 2011 by orbiter | fix for oob in snippet generation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7425 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java |
Tue Jan 04 13:24:41 CET 2011 by orbiter | npe assert fix git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7424 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/ReferenceContainerCache.java |
Tue Jan 04 01:44:03 CET 2011 by orbiter | added target that compiles only the cora package git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7423 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.xml |
Tue Jan 04 00:06:33 CET 2011 by orbiter | fix for http://forum.yacy-websuche.de/viewtopic.php?p=21489#p21489 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7422 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/table/Table.java |
Mon Jan 03 23:53:15 CET 2011 by orbiter | added JSON parser code from json.org (added generics to it) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7421 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/JSONArray.java, source/net/yacy/cora/document/JSONException.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/document/JSONTokener.java |
Sun Jan 02 22:17:21 CET 2011 by lotus | UPnP: map port again if we are not reachable (e.g. when router rebooted) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7419 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/yacy.java |
Sun Jan 02 21:38:01 CET 2011 by lotus | same units for memory observer configuration (MiB) old setting for DHT (RAM) will be lost after update can be set on /Performance_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7418 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init, htroot/PerformanceQueues_p.java, htroot/Performance_p.html, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/search/SwitchboardConstants.java, source/net/yacy/kelondro/util/MemoryControl.java |
Thu Dec 30 02:46:37 CET 2010 by orbiter | enhancement in table display of path names git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7417 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js |
Thu Dec 30 02:04:08 CET 2010 by f1ori | * fix layout of yacyinteractive git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7416 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacyinteractive.html |
Thu Dec 30 01:49:51 CET 2010 by orbiter | layout git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7415 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js |
Thu Dec 30 01:02:42 CET 2010 by orbiter | maybe found bug in non-working index dumper git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7414 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Wed Dec 29 19:56:42 CET 2010 by orbiter | added a (temporary) description for 27c3 search set-up that everyone can see if they use YaCy here. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7413 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacyinteractive.html |
Wed Dec 29 15:51:07 CET 2010 by orbiter | fix for NPE and added comment in search result git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7412 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, source/de/anomic/search/RankingProcess.java |
Wed Dec 29 15:30:25 CET 2010 by orbiter | added search suggestions for interactive search: is only shown if there are no search results git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7411 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Crawler_p.java, htroot/suggest.java, htroot/yacyinteractive.html, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/Switchboard.java |
Wed Dec 29 02:54:27 CET 2010 by orbiter | - moved some variables from Stwitchboard to new class AccessTracker - added a limitation in access tracking to delete queries which are older than 10 minutes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7410 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/AccessTracker_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/AccessTracker.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/table/Table.java |
Tue Dec 28 18:59:14 CET 2010 by f1ori | * labels for index delete checkboxes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7409 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexControlRWIs_p.html |
Tue Dec 28 18:25:39 CET 2010 by f1ori | * fix negative filelength for files >2G git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7408 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/Response.java, source/net/yacy/cora/protocol/HeaderFramework.java |
Tue Dec 28 17:44:55 CET 2010 by orbiter | fix for really bad table iteration implementation: reduction of IO git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7407 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/yacyNewsPool.java, source/net/yacy/kelondro/table/Table.java |
Tue Dec 28 14:04:48 CET 2010 by low012 | *) Ooops... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7406 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/mmParser.java |
Tue Dec 28 14:03:49 CET 2010 by low012 | *) minor enhancements git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7405 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/sidAudioParser.java |
Tue Dec 28 13:10:48 CET 2010 by orbiter | no db lookup during search / avoid IO git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7404 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearchitem.java |
Tue Dec 28 13:06:04 CET 2010 by low012 | *) added SID file (Commodore 64) sound file parser *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7403 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/httpd.mime, source/net/yacy/document/Classification.java, source/net/yacy/document/Document.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/sidAudioParser.java |
Tue Dec 28 03:15:22 CET 2010 by f1ori | * fix filesize in ftp crawls git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7402 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/Response.java |
Tue Dec 28 02:57:05 CET 2010 by orbiter | more small bugfixes and light refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7401 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, htroot/yacysearch.java, source/de/anomic/yacy/yacyNewsQueue.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/kelondro/blob/HeapReader.java |
Tue Dec 28 02:27:36 CET 2010 by orbiter | remove '*' from query string (people believe that this is a wild card) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7400 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/yacysearch.java |
Tue Dec 28 00:57:29 CET 2010 by orbiter | patch for http://forum.yacy-websuche.de/viewtopic.php?p=21460#p21460 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7399 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/Condenser.java |
Tue Dec 28 00:11:56 CET 2010 by orbiter | - allow only scanned resourced with granted status - increased time-out when scanning an ip range git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7398 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Scanner.java |
Mon Dec 27 21:13:31 CET 2010 by low012 | *) added FreeMind (http://freemind.sourceforge.net/) mindmap parser *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7397 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/httpd.mime, source/net/yacy/document/Classification.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/swfParser.java |
Mon Dec 27 16:53:41 CET 2010 by low012 | *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7395 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/zipParser.java |
Mon Dec 27 00:21:34 CET 2010 by orbiter | small changes to scanner git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7393 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, htroot/js/yacyinteractive.js |
Sun Dec 26 17:25:17 CET 2010 by orbiter | enhanced network scanner (less name resolving during scanning and no name resolving during search) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7392 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java |
Wed Dec 22 01:07:46 CET 2010 by orbiter | added skin for 27c3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7391 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: skins/27c3.css |
Wed Dec 22 01:04:30 CET 2010 by orbiter | fix for http://forum.yacy-websuche.de/viewtopic.php?p=21439#p21439 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7390 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Wed Dec 22 00:48:05 CET 2010 by orbiter | - fix for map view - remove some UNRESOLVED PATTERN - maybe a fix for non-flushing cache git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7389 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/PerformanceMemory_p.java, htroot/Status.html, htroot/yacysearch.html, htroot/yacysearch_location.html, htroot/yacysearchtrailer.html, source/net/yacy/kelondro/rwi/IndexCell.java |
Mon Dec 20 23:51:50 CET 2010 by orbiter | - new java version 1.6 - replaced old gif animator by java 1.6 gif animator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7388 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: build.properties, source/de/anomic/search/RankingProcess.java, source/net/yacy/visualization/AnimationGIF.java |
Sat Dec 18 11:22:54 CET 2010 by orbiter | fix to ftp loader git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7387 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/FTPLoader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java |
Sat Dec 18 01:44:19 CET 2010 by orbiter | another fix to the ftp crawler: now correct directory listings according to rfc2640 (path with spaces) and better title names for such files git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7386 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/parser/genericParser.java |
Fri Dec 17 01:52:24 CET 2010 by orbiter | more fixes to smb crawling: better file names git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7384 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/SMBLoader.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/torrentParser.java |
Fri Dec 17 01:11:42 CET 2010 by orbiter | several fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7383 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/SMBLoader.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSMessage.java |
Fri Dec 17 00:47:36 CET 2010 by orbiter | fix for token parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7382 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Thu Dec 16 19:51:20 CET 2010 by low012 | *) Urks... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7380 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/cytag.java |
Thu Dec 16 12:19:22 CET 2010 by orbiter | fix for npe git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7379 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/server/serverObjects.java |
Thu Dec 16 03:15:20 CET 2010 by orbiter | redesigned CrawlStartScanner user interface and added more features: - multiple hosts for environment scans can be given (comma-separated) - each service (ftp, smb, http, https) for the scan can be selected - the scan result can be accumulated or refreshed each time a network scan is made - a scheduler was added to repeat a scan and add all found urls to the indexer automatically git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7378 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, htroot/api/bookmarks/get_bookmarks.java, source/net/yacy/document/parser/genericParser.java |
Thu Dec 16 01:18:05 CET 2010 by low012 | *) cleaning up the code a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7377 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CookieTest_p.java, htroot/CrawlResults.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/feed.java, htroot/cytag.java, source/de/anomic/yacy/yacyChannel.java |
Wed Dec 15 22:39:02 CET 2010 by f1ori | * deny adminForLocalhost and intranet network setup also on bootup and not only on network switch * require authentication for yacybot what ever adminForLocalhost is set to (after this patch, is the rule from above really nesseccary, the crawler also checks the robots.txt) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7376 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/Switchboard.java |
Tue Dec 14 13:12:13 CET 2010 by orbiter | enhanced speed and usability of network scanner servlet git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7373 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, skins/generic_pd.css, skins/pdblue.css |
Tue Dec 14 02:14:15 CET 2010 by orbiter | - removed old intranet scanner (the generic scanner now completely subsumes the old one) - added information about granted access - enhanced servlet design - added submit-feedback (because it is a long-running task) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7372 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/kelondro/data/meta/DigestURI.java |
Mon Dec 13 19:19:37 CET 2010 by orbiter | added new network scanner - you can scan any ip or host in the internet for services - this replaces the intranet scanner git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7371 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CrawlStartIntranet_p.java, htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, htroot/env/templates/submenuIndexCreate.template, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/protocol/Scanner.java |
Sat Dec 11 16:23:35 CET 2010 by low012 | *) Fixed links to CrawlStart-page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7370 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.html, htroot/Status.html, locales/de.lng |
Sat Dec 11 10:21:37 CET 2010 by orbiter | fixes to ftp client git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7369 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java |
Mon Dec 06 12:09:16 CET 2010 by f1ori | * use local backup-file, if remote network-definition is not availible * resolve single point of failure in networks, managed by central network-definitions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7363 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/util/FileUtils.java |
Thu Dec 02 13:19:59 CET 2010 by orbiter | - better caclculation of search result size - predefined search recommendations git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7361 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/SearchEvent.java |
Thu Dec 02 09:08:01 CET 2010 by orbiter | added a directory-structure reader to ftp client git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7359 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/YaCySearchClient.java, source/net/yacy/cora/protocol/ftp/FTPClient.java |
Wed Dec 01 15:55:40 CET 2010 by orbiter | fixed crawl queue delete function git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7357 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexCreateWWWLocalQueue_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/NoticedURL.java |
Wed Dec 01 15:43:07 CET 2010 by orbiter | enhanced interactive search: - better table design - less enumeration of same table structure (prepared now for streaming) - added a 'remove filetype' link git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7356 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/js/yacyinteractive.js, htroot/yacyinteractive.html |
Mon Nov 29 21:09:53 CET 2010 by lotus | do not reset language on config change git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7354 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.java |
Mon Nov 29 20:49:57 CET 2010 by orbiter | one more fix to Domains git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7353 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/Domains.java |
Mon Nov 29 20:32:47 CET 2010 by orbiter | fix for latest commit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7352 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/Domains.java |
Mon Nov 29 20:18:44 CET 2010 by orbiter | implemented local host address discovery as posted in http://forum.yacy-websuche.de/viewtopic.php?p=21310#p21310 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7351 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/protocol/Domains.java |
Mon Nov 29 09:41:29 CET 2010 by orbiter | fix for ftp client: suppress bad directory listing time-out git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7348 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java |
Mon Nov 29 08:52:10 CET 2010 by orbiter | removed warnings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7347 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Connections_p.java, htroot/yacysearch.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/yacy/dht/Transmission.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/cora/protocol/http/HTTPConnector.java |
Sun Nov 28 04:05:22 CET 2010 by low012 | *) Fixed typo. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7346 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/yacy/dht/Dispatcher.java |
Sat Nov 27 10:16:16 CET 2010 by orbiter | - view image files from cache - fixed generic header settings; affects CORS functionality git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7344 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/CacheResource_p.java, htroot/ViewFile.html, htroot/ViewFile.java, htroot/suggest.java, htroot/yacysearch.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java |
Fri Nov 26 23:38:08 CET 2010 by low012 | *) HTML fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7342 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/BlacklistTest_p.html |
Fri Nov 26 23:19:31 CET 2010 by low012 | *) HTML fixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7341 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Collage.html, htroot/WatchWebStructure_p.html |
Fri Nov 26 12:15:42 CET 2010 by orbiter | removed two warnings and reverted one change git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7340 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/FilterEngine.java |
Thu Nov 25 22:16:47 CET 2010 by apfelmaennchen | various small changes to ymarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7339 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, htroot/Table_YMark_p.java, htroot/api/ymarks/import_ymark.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java |
Thu Nov 25 00:40:55 CET 2010 by apfelmaennchen | some code clean up for the last post git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7338 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, source/de/anomic/data/YMarkTables.java |
Thu Nov 25 00:26:29 CET 2010 by apfelmaennchen | another step towards an auto tagging function for YMarks git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7337 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarkWordCountComparator.java |
Wed Nov 24 00:10:45 CET 2010 by low012 | *) hopefully fixed bug described in http://www.yacy-forum.org/viewtopic.php?f=12&t=385 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7335 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Status.java |
Mon Nov 22 20:12:51 CET 2010 by f1ori | * make .yacy-domains crawlable (.yacy-domains are local domains, so only in custom networks/peers) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7334 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDemon.java |
Mon Nov 22 00:55:40 CET 2010 by f1ori | * fix small bug in sessionid-removal * add testcase for seesionid-removal git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7333 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, test/net/yacy/cora/document/MultiProtocolURITest.java |
Sun Nov 21 23:46:12 CET 2010 by f1ori | * don't index pages, which redirect, twice * there fore auto-redirection of HTTPClient for crawling is disabled and the old code is reactivated git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7332 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/crawler/retrieval/HTTPLoader.java |
Sun Nov 21 01:41:45 CET 2010 by low012 | *) Ooops! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7329 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Wiki.java |
Sun Nov 21 01:39:21 CET 2010 by low012 | *) renamed classes according to standard Java coding conventions *) removed unsused code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7328 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/wiki/AbstractWikiParser.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/data/wiki/WikiParser.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/importer/MediawikiImporter.java |
Sun Nov 21 01:28:32 CET 2010 by low012 | *) removed unsused code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7327 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: |
Sun Nov 21 01:13:08 CET 2010 by low012 | *) trying to make this class less confusing (probably failing) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7326 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/de/anomic/data/wiki/wikiCode.java |
Wed Nov 17 16:17:29 CET 2010 by apfelmaennchen | intermediate step for a YMark auto-tagging function based on word frequencies. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7325 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, source/de/anomic/data/YMarkKeyValueEntry.java, source/de/anomic/data/YMarkTables.java |
Tue Nov 16 01:48:38 CET 2010 by apfelmaennchen | added a drill-down for metadata and word count to /api/ymarks/test_treeview.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7324 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.java, htroot/api/ymarks/get_treeview.java, source/de/anomic/data/YMarkTables.java |
Mon Nov 15 15:47:16 CET 2010 by f1ori | * fix bug when browsing local filesystem (e. g. repository) with yacy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7323 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/document/parser/html/ContentScraper.java |
Mon Nov 15 02:48:54 CET 2010 by mikeworks | de.lng: Started translation of YMark feature here http://localhost:8080/Table_YMark_p.html Table_YMark_p.html: removed some labels because columns is not filled yet and added ids for other label references and added full enabled="enabled" tags because SHORTTAG is not specified in XHTML 1.0 Strict git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7322 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, locales/de.lng |
Sat Nov 13 15:27:30 CET 2010 by lotus | change language immediately there is a bad behaviour: if language is changed, servlets are translated but the returning servlet is yet the old one. therefore the page is loaded again after language changed. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7321 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/ConfigBasic.html, htroot/ConfigBasic.java |
Sat Nov 13 14:02:56 CET 2010 by apfelmaennchen | enabled rebuildIndex for /Table_YMark_p.html (rebuilds the tags and folders index) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7320 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, htroot/Table_YMark_p.java, source/de/anomic/data/YMarkIndex.java |
Sat Nov 13 11:32:37 CET 2010 by apfelmaennchen | enabled YMark Import for /Table_YMark_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7319 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, htroot/Table_YMark_p.java, htroot/api/ymarks/import_ymark.html, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/info.html |
Fri Nov 12 22:13:27 CET 2010 by apfelmaennchen | slightly enhanced interface for /Table_YMark_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7318 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/Table_YMark_p.html, htroot/Table_YMark_p.java |
Fri Nov 12 17:02:20 CET 2010 by sixcooler | using less memory on merging and rewriting blobs git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7317 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/kelondro/blob/ArrayStack.java |
Mon Nov 08 10:37:44 CET 2010 by f1ori | * fix FilterEngine, forgot comparision-operator git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7314 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: source/net/yacy/repository/FilterEngine.java |
Sun Nov 07 21:03:01 CET 2010 by apfelmaennchen | update to ymarks - get_xbel.xml is almost working - startet ymark api documentation info.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7313 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/info.html, htroot/api/ymarks/test_import.html, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksXBELImporter.java |
Sat Nov 06 17:50:33 CET 2010 by f1ori | * add domaincheck (local/global/domainlist) to urlcleaner git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7311 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: htroot/IndexCleaner_p.java, source/de/anomic/search/MetadataRepository.java |
Sat Nov 06 16:09:28 CET 2010 by f1ori | * %0 does not belong to the IPv6-Address -> entry does not work on some systems git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7310 6c8d7289-2bf4-0310-a012-ef5d649a1542 Changed Files: defaults/yacy.init |