Issues with Malwarebytes and POGS
Over the last few weeks I've seen several forum threads concerning issues with a new update of Malwarebytes interfering negatively with POGS.
If anyone is currently having issues, please see this thread for advice: https://pogs.theskynet.org/pogs/forum_thread.php?id=827&postid=5310#5310.
The current going solution is to update Malwarebytes again to get past the version that was causing POGS issues.
Happy Crunching! 17 Oct 2017, 23:53:51 UTC · Comment
Sourcefinder - Real data coming soon!
I've made a fairly important announcement about the future of Sourcefinder, our other project.
There's a high chance that we'll be processing data from ASKAP in the future!
Anyone who's interested, please take a look at this post. 21 Jun 2017, 4:55:43 UTC · Comment
Minor planet named Bernard
A minor planet in the Solar System will officially be known as Bernardbowen from today after Australian citizen science project theSkyNet won a competition to name the celestial body.
The minor planet was named by the International Centre for Radio Astronomy Research (ICRAR) in honour of their founding chairman Dr Bernard Bowen.
Bernardbowen sits in the asteroid belt between Mars and Jupiter and takes 3.26 Earth years to orbit the Sun.
The minor planet was discovered on October 28, 1991, and until now has been known as (6196) 1991 UO4.
Based at ICRAR, theSkyNet has been running since 2011 and sees citizen scientists donating their spare computing power to help Australian astronomers uncover the mysteries of the Universe.
Its 50,000-odd volunteers entered an International Astronomical Union (IAU) contest to name planets beyond our Solar System.
Project founders ICRAR also won the right to name a minor planet within our Solar System.
Bernardbowen was one of 17 minor planets to be christened today.
Other newly named minor planets include Kagura, after a traditional Shinto theatrical dance, and Mehdia, which is equivalent to the Arabic word for gift.
Dr Bowen is renowned as one of the country’s finest science administrators and has presided over scientific advances ranging from the oceans to the skies. He was instrumental in the establishment of ICRAR in 2009, and helped bring part of the Square Kilometre Array telescope to Western Australia.
A full list of the citation of the minor planets can be found at the IAU Minor Planet Circular.
Bernardbowen can be found on the Minor Planet Centre site, including an interactive showing its position in the Solar System. 17 Feb 2017, 6:44:38 UTC · Comment
Sourcefinder Beta - Attempt 2
Earlier today we were having some issues with the hard disk size of the sourcefinder server. While I was attempting to increase the disk size, I accidentally terminated the amazon instance causing the server to be destroyed.
After a large amount of facepalming and re-installing, the server is back up with a larger hard disk at the following address: http://126.96.36.199/duchamp/.
Unfortunately all of the forum posts have been deleted along with the server, and there may be a few latent setup issues that I haven't addressed quite yet.
Before accidentally destroying everything, I did copy off some of the bug reports that people had already made in to a Trello board. This board will store all of the current issues that I'm investigating, so you can all keep track of what I'm doing. https://trello.com/b/Y3XkcXMJ/sourcefinder-public. Once I've read a bug report on the forums, I'll update that board with the relevant details and get to work on fixing it.
I also plan on posting a weekly changelog of everything I've managed to fix so far.
I'm uploading a 30GB packet of work units to the server for everyone to work on. I'll get them sent the moment I can.
I'm truly sorry about all of this, and I hope everyone will still be happy enough to participate in sourcefinder.
Sam 9 Feb 2017, 7:33:42 UTC · Comment
Merry Christmas / Happy Holidays!
It's certainly been a long year, and a little rocky for theSkyNet, but we're slowly bringing things back on track.
I would like to announce that work on Sourcefinder has come to the stage where it's ready to go into beta, so look out for a Sourcefinder announcement in the new year!
Also I want to say thank you to everyone for volunteering their computer time to POGS, everyone on theSkyNet team greatly appreciates the work you do for us :)
So I'll close with a big Merry Christmas and a Happy New year to all of you from theSkyNet team. If you don't celebrate Christmas, Happy Holidays to you instead! 24 Dec 2016, 5:19:00 UTC · Comment
Spam prevention measures.
16 Nov 2016, 1:48:23 UTC
An extremely large number of spam users and teams were detected on the POGS forums recently. By extremely large, I mean more than 90% of the registered accounts and teams were spam.
Preventative measures have been put in place to increase the difficulty for spammers creating spam accounts and teams.
From now on, every profile modification requires a reCAPTCHA to be filled in (one of the simple ones that only requires you to tick a box :P) and the stopforumspam.com service is also being utilised to prevent the creation of spam accounts using known spam emails and IP addresses.
Additionally, a very large number of user accounts and teams are being deleted.
Accounts will be deleted if they:
Have a profile containing a link and
Have no forum posts and
Have no registered hosts.
Teams are deleted if they:
Have 0 or 1 members and
Have 0 total credit and
Are not BOINC-Wide teams and
Have descriptions containing a link and
The owner has no hosts and no posts.
Keep in mind, these are ANDs not ORs. Accounts and teams are only deleted if they fulfill ALL of the criteria listed above.
16 Nov 2016, 1:48:23 UTC
Do you have a Question for Steve Wozniak?
This is primarily for the people of Perth, but if you'd like me to ask Steve Wozniak a question - complete the survey - www.surveymonkey.com/r/QuestionForWozPerth 10 Aug 2016, 0:34:14 UTC · Comment
How cool is this!!!
I’m excited to announce that I am hosting Think Inc.’s event An evening with Steve Wozniak Perth on 24th August. I’ll be talking to Woz about Apple, innovation, entrepreneurship, virtual reality and the amount of grunt needed from Astronomy. 5 Aug 2016, 6:39:32 UTC · Comment
Sorry for the radio quiet
Sorry for the quietness of late. I've been rather poorly.
But back on deck this week and we've got 64,000 new galaxies to process. 2 Aug 2016, 4:01:54 UTC · Comment
More Galaxies are on the way
More galaxies are on the way 9 May 2016, 0:47:51 UTC · Comment
Comparing galaxy morphology at high and low redshift
Hopefully everyone has been enjoying the latest set of galaxies. These objects are galaxy mergers identified by citizen scientist volunteers at galaxyzoo. We are going to be using them as part of an experiment contrasting different indicators of galaxy morphology at high and low redshift, so early and late epochs in the universe, respectively. A control sample of non-mergers is also being run at lower priority. In particular, working with my colleagues at JHU, we are measuring what is called non-parametric morphological indicators -- things like concentration, asymmetry, Gini coefficient, and many more -- both on the observed telescope images and on the physical parameter maps your CPUs are generating. This is normally done only on the observed images, and by working with derived physical quantities we hope to eliminate potential bias. Also, such measurements (for a complete set of non-parametric indicators) have never yet been compiled for the low redshift universe, though done frequently at high z. By working with the galaxies you are now processing we will establish a 'baseline' for galaxy morphology in the present day universe. Thanks! 29 Feb 2016, 15:09:33 UTC · Comment
I've just load 1200 new galaxies when are experiments on GalaxyZoo mergers 22 Feb 2016, 3:10:49 UTC · Comment
Out of galaxies
We've run out of galaxies for a bit - more coming soon 15 Feb 2016, 4:52:11 UTC · Comment
You helped name an exoplanet!
Some great news announced yesterday by the International Astronomical Union - theSkyNet's name suggestion was selected for an exoplanet!
The planet around the star Pollux is now named Thestias thanks to djrichmatthews' suggestion. All the final names and the statistics can be seen on the official IAU webpage http://nameexoworlds.iau.org/names.
You might notice that Thestias is suspiciously NOT the name we submitted (which was Leda). This is because Leda is the name of both a moon around Jupiter and an asteroid in the Solar System, so the IAU asked if we'd mind suggesting something else that's related to Pollux and Leda from Greek Mythology. Thestias is Pollux's grandfather (Leda's Dad) so it's a fitting change.
We're SO EXCITED with this result, congratulations to everyone who suggested, voted and was involved, it was a true team effort and we're so pleased that your name was chosen.
-Kirsten and theSkyNet team 17 Dec 2015, 4:18:23 UTC · Comment
Yesterday we processed the 25 millionth area of POGS.
The two lucky crunchers will be getting Lego and T-shirts 8 Dec 2015, 1:14:10 UTC · Comment
Server Crash and New Galaxies
As lots of you noticed, we had a fairly major crash over the weekend. The AWS hardware that the POGS server sits on had a complete failure and a we had to kill and rebuild the whole server.
Fortunately with the magic of AWS and a fairly conservative backup scheme this only caused moderate cursing from theSkyNet team.
This also coincided with a whole swathe of new galaxies for you to crunch. With these new galaxies we're also looking at radial flux, which I'm told adds another dimension to the data we're gathering :)
Also anyone that was still using the old master URL: http://ec2-23-23-126-96.compute-1.amazonaws.com/pogs/login_form.php?next_url=home.php
Will need to switch over the new and shiny master url: http://pogs.theskynet.org/pogs/
As always thanks so much to everyone for stick through with us during the down times, we truly value all our amazing volunteers.
Alex 28 Aug 2015, 1:21:13 UTC · Comment
New area types and galaxies
50 new galaxies are going out for processing today containing two new types of areas: integrated flux areas and radial areas.
More galaxies with these new area types will be on the way soon! 21 Aug 2015, 3:05:31 UTC · Comment
Run out of Galaxies
We've run out of galaxies to process at the moment. More are on the way. 21 Jul 2015, 0:06:17 UTC · Comment
Debugging Size Classes Take II
Just a quick heads up - David Anderson (the guy who wrote most of the BOINC server) from Berkeley is currently having another crack at debugging the size class problem we have. 7 Jul 2015, 3:35:27 UTC · Comment
Help name exoplanets!
Hi theSkyNet POGS volunteers,
The International Astronomical Union is allowing astronomical organisations (like theSkyNet) to submit potential names for 20 exoplanet systems outside our Solar System.
We get to submit a name, and we'd like your help to choose it.
If you get suggesting with names (that follow the IAU rules, see below) on June 10th we'll compile all the suggestions and take a vote for what name to submit to the IAU on June 15th!
Put your suggestions in replies on this thread on the message boards. Don't forget to tell us which system your name suggestion is for, see the list linked below (and if you've got a reason why you think the name is good, then add that in too!)
The name must be 16 characters or less in length, preferably one word, pronounceable, non-offensive and not too similar to an existing name of an astronomical object (you can check your suggested name at Sesame Service (http://cds.u-strasbg.fr/cgi-bin/Sesame) or the Minor Planet Centre (http://www.minorplanetcenter.net/db_search)).
The names must not include names of pet animals, names of a purely or principally commercial nature, names of individuals, places or events principally known for political, military or religious activities, names of living individuals or names protected by trademarks or other forms of intellectual property.
More details on the rules are on the IAU website: http://www.nameexoworlds.org/the_process.
So is a list of the 20 exoplanet systems that are up for naming: http://www.nameexoworlds.org/the_exoworlds.
Happy suggesting! I'm looking forward to seeing what you come up with. 2 Jun 2015, 7:19:29 UTC · Comment
Debugging size classes
Just a quick heads up - David Anderson (the guy who wrote most of the BOINC server) from Berkeley is currently trying to debug the size class problem we have.
Hopefully this should affect people, but "famous last words...." 28 May 2015, 5:00:17 UTC · Comment
We've found a problem with the BOINC size class code. I currently have 2,500,000 inactive WUs and only 20 unsent
I'm working with the BOINC developers to fix the problem. In the mean time I'm turning off the size class code 21 May 2015, 1:33:13 UTC · Comment
Things we're changed/working on
29 Apr 2015, 9:49:45 UTC
A lot of the Android clients aren't downloading the new version of the code - they are still running the older 3.41 version.
If you have one please could you trigger an update of the code 28 Apr 2015, 2:08:29 UTC · Comment
I'm seeing quite a lot of errors from Android users. It looks like the client hasn't downloaded the PIE version. I'm investigating 27 Apr 2015, 3:54:25 UTC · Comment
We have 132 new galaxies to test things out 27 Apr 2015, 3:50:20 UTC · Comment
BETA testers wanted
We have the beta test server running. If people are willing to help test stuff (without receiving credit - Sam is working on the new credit model) connect to
It has 17 filter bands (GALEX, Pan-STARRS, SDSS, WISE, Herschel, Spizter) and new wrappers and new Android. I need to make sure the amount of disk I allow is enough for the new models.
Unfortunately the server will be shutdown once the testing is complete; so you won't get any credits for helping (sorry), but you will get my thanks. 13 Apr 2015, 2:36:46 UTC · Comment
BETA testers wanted
We have the beta test server running. If people are willing to help test stuff (without receiving credit - Sam is working on the new credit model) connect to.
It has 17 filter bands (GALEX, Pan-STARRS, SDSS, WISE, Herschel, Spizter) and new wrappers and new Android. I need to make sure the amount of disk I allow is enough for the new models.
Unfortunately the server will be shutdown once the testing is complete; so you won't get any credits for helping (sorry), but you will get my thanks. 13 Apr 2015, 2:36:46 UTC · Comment
Good news everybody!
We've got some new data from our friends at Johns Hopkins with a whole raft new improvements and additions.
What does this mean?
We still have to test the new data with the new clients to make sure everything goes according to plan which we would love your help with. First we’ll do some in house testing over the weekend but (hopefully) on Monday I’ll be putting out a call to volunteers to help us test some of this new data, including the brand new android 5 client.
Note: due to the setup of the test server we won’t be maintaining credit which may or may not disappear without warning. 10 Apr 2015, 6:56:52 UTC · Comment
11 Mar 2015, 1:29:27 UTC
Sorry folks - David is having a few problems with the new galaxies - we hope to have things flowing shortly. But he is doing the following to the new data which is pretty tricky:
(a) adaptive smoothing across all bands, after initial resolution match (this next run will be Pan-STARRS & SDSS, then we'll add GALAX, WISE, Spitzer & Herschel)
(b) foreground stellar masking
(c) segmentation map defining the main galaxy in each input, derived from a "white-light" image
(d) integrated SED placed artificially into pixel (1,1).
Also the new Web Site Security Certificate has been deployed.
The Android code is about to be loaded onto the test server so we hope to release it asap.
11 Mar 2015, 1:29:27 UTC
Run out of Galaxies
We'll run out of Galaxies over the weekend.
Next week we'll:
6 Mar 2015, 5:10:52 UTC
POGS can’t currently support Lollipop, but will be able to soon, we’ll (Alex) provide updates as soon as possible. 12 Feb 2015, 4:19:19 UTC · Comment
Happy Australia Day 26 Jan 2015, 1:01:54 UTC · Comment
theSkyNet POGS project Update
We’re planning an upgrade for POGS in the next few months that will see us add more calculations to the suite already conducted on each galaxy image, add more data types (such as higher resolution images from the WISE telescope, http://www.nasa.gov/mission_pages/WISE/main/, which collects infrared light) and much, much more.
There are some big changes, and some smaller ones, included in the upgrade. We’ve given a short summary of everything below, but if you’d like any more specific detail or have questions then please let us know.
As part of the upgrades we intend to run some simulated galaxies through the upgraded POGS code as a check on the accuracy of our the data processing. These 'fake' galaxy images are created by supercomputers as part of a large cosmological simulation http://www.illustris-project.org. Once the simulated galaxies are processed by the upgraded POGS ‘spectral energy distribution’ fit code, we can compare the results to the known properties used when creating the fake (but highly realistic) galaxies to confirm the upgraded code works as expected and is producing accurate science results for the astronomers.
We’re also upgrading the data retrieval pipeline, continuing to get enhanced infrared images from the WISE telescope for all galaxies that have already been processed, as well as some more galaxy data from other new sources. This new data will broaden our results on the galaxies that have been already processed - helping the astronomers to see more of the bigger picture, so to speak.
Kevin and the team will also upgrade the input data processing pipeline as well. This will include improving the pre-processing of input images to enhance their usefulness in especially faint regions (like the outer parts of a galaxy). A technique called adaptive smoothing will be used to achieve this result without blurring of bright regions. They will also develop ‘parameter prior cubes’, sets of position-dependent bracketing parameter values that should be used in SED fitting calculations. You can think of them as educated guesses for each parameter. This will stop us calculating every possible combination of parameters on every galaxy — saving time and resources. This also means we will be changing the credit system from the current points-per-pixel based model, as with ‘priors’ all pixels are no longer equal. We will let you know when the credit change will occur and will do our best to make sure it’s equitable and results in approximately the same amount of credit you currently earn.
We will also be upgrading the output processing to produce better results images. This will include things like stellar masking (from catalogs, or directly from the images) to remove foreground stars from the calculations, and we also start using a code called ‘galfit’ (http://users.obs.carnegiescience.edu/peng/work/galfit/galfit.html) to give a mass surface density of the galaxies (a new calculation) and also try another code called ‘CIGALE’ (http://cigale.lam.fr/) and others on the data.
All of this comes together to mean better science output from POGS - something we’re really keen on and we hope you are too!
--TheSkyNet Team 8 Dec 2014, 2:47:51 UTC · Comment
Here come the Galaxies
To misquote the Aussie band "Rogue Traders"
Here come the galaxies, here come the galaxies
Another 8000 loaded ready to be processed. 20 Nov 2014, 4:42:57 UTC · Comment
Pause in work-units
We'll be halting the WUs shortly whilst I upgrade the servers and the software. 4 Nov 2014, 2:54:02 UTC · Comment
Our 50,000th Galaxy
We've designed a special trophy for those who process our 50,000th galaxy.
If you want to see the galaxy it is here. 23 Sep 2014, 9:19:34 UTC · Comment
TheSkyNet turns 3
Last Saturday (the 13th) was our third birthday. It really snuck up on us, we can't believe it's been three years already since our first members signed up! To celebrate we've created something special to share with you all.
Alex (Eckley) has created this amazing gigapixel mosaic of ALL the results you've helped create on theSkyNet POGS. He included results from over 45,000 galaxies (in fact we're processing the 50,000th galaxy this week!) Have an explore of the image, we recommend viewing at fullscreen.
This photomosaic is of the results from processing an image of M100 - a relatively nearby galaxy. This image is the 'sfr' result image, or star formation rate - how fast new stars are being born in different parts of the galaxy. The overall image is made up as a mosaic of results files from theSkyNet POGS, over the many different properties you help us work out for each galaxy.
This is such an impressive picture showing the incredible volume of results you've helped produce since theSkyNet began.
Thank you to each and every one of our members for their support, as always we can't do it without you.
We're off to explore this image a bit more, and to see if we can find the mini image of M100 within the mosaic!
-TheSkyNet Team 17 Sep 2014, 1:28:12 UTC · Comment
We fairly certain that we've tracked down and fixed the issue that was causing the 184 error on new windows and boinc clients when trying to connect to the pogs server. If anyone is still getting the error please let me know.
Alex 2 Sep 2014, 8:09:09 UTC · Comment
Pause in Galaxies
We'll have a pause of a day or so as we're having some problems with the production pipeline.
Please bear with us. 6 Aug 2014, 8:20:45 UTC · Comment
Pause in Galaxies
They'll be a pause in the galaxy flow for a few days so I can build the latest BOINC server, apply the latest security patches, etc. 16 Jul 2014, 7:27:52 UTC · Comment
We've just released another 2,000+ galaxies 21 Jun 2014, 1:02:48 UTC · Comment
I'll be loading new galaxies tomorrow - we'll run out during today 20 Jun 2014, 4:04:36 UTC · Comment
We're going to let the current galaxies run out - then do an upgrade of the BOINC software next week.
So once the queue is empty no more work units for a week. 9 Jun 2014, 2:47:01 UTC · Comment
BOINC pentathlon Recap
Wow what an amazing two weeks. A big thank you to the Team at SETI.Germany for hosting an amazing event, and secondly a massive thank you to everyone how contributed to theSkyNet POGS as part of the 5th BOINC Pentathlon, special, one of a kind trophies will be making their way to your trophy cabinets (on theSkyNet.org) soon. More details on the prizes will follow shortly.
Congratulations to the winners Planet 3DNow! Who put in a truly galactic effort processing 37,617,784 credits across the two week challenge.
As promised I put together some stats about the challenge, if there is something else you are wondering about let me know and I’ll see if I can figure it out.
Participation rates during the Pentathlon:
Credits awarded on theSkyNet POGS during the Pentathlon:
TFLOPS on theSkyNet POGS during the Pentathlon:
19 May 2014, 9:11:54 UTC
There will be an outage of an hour or two on Tuesday 20th May from 9:30 Western Australian Time.
I'll be upgrading the kernels on machines, installing security patches and increase the database size and performance. 18 May 2014, 1:46:08 UTC · Comment
BOINC Pentathlon Lego Prizes!!
The BOINC pentathlon has chosen theSkyNet POGS as their marathon project this year - we're very excited that SETI-Germany and the team behind the pentathlon chose us and so to celebrate we thought we'd add a bit more to the fun.
A word association game has been started on the POGS forum, so jump on and get involved. At the end of the pentathlon we'll randomly choose two winners that have participated in the thread and are part of the pentathlon.
The prizes will be a $60 AUD Lego pack sent anywhere in the world (~40 Euros or $56 USD - thanks to pschoefer for the conversions). The winners get to choose whatever Lego they'd like - jump on the Australian Lego store and see what you might choose if you win!
Looking forward to seeing all your entries in the game.
reposted from https://www.theskynet.org/news/53 8 May 2014, 1:08:29 UTC · Comment
TheSkyNet POGS has been chosen as the marathon project in the BOINC pentathlon! To celebrate, we've got some great prizes to give away - including some free Lego. More details to come on how these prizes will be given out to pentathlon participants. Thanks for being part of theSkyNet!
-theSkyNet 6 May 2014, 1:59:07 UTC · Comment
Please don't use the custom clients - they don't always get the right answer.
I hope to build a release optimised clients after easter and we'll start testing the GPU version soon. 9 Apr 2014, 9:00:17 UTC · Comment
The upgrade is complete - let me know if you encounter any BOINC related problems. 25 Mar 2014, 3:36:14 UTC · Comment
Good Bye Spam
We've set up the server with shiny new spam filter, lets see how long this one lasts.
Alex 24 Mar 2014, 1:55:15 UTC · Comment
We'll be off the air for a day or so next week as we need to upgrade all the BOINC server software to the latest versions.
We'll also be installing a spam filter as we're starting to get spam postings. 19 Mar 2014, 23:00:30 UTC · Comment
Minor down time
Just a quick heads up that there will be up to an hour of down time on the POGS server today as we add a new database table. The new table will improve the performance of galaxy searches and allow us to award trophies based on the number of galaxies a user has helped contribute to.
27 Feb 2014, 1:42:15 UTC · Comment
Galileo Galilei - Challenge Recap!
Wow! What an amazing week for theSkyNet and POGS. In the last 7 days we’ve achieved a whole host of project milestones. First of all a big thank you to the ‘Astronomy.Ru Forum’, for setting up another exciting challenge and secondly to everyone who connected their computer to the POGS network, we could not do this without you.
I’ve put up a post with lots of cool stats and the prizes winning results, over on the main page http://www.theskynet.org/news/49. It would be really awesome if you could check it out.
Alex 24 Feb 2014, 5:22:06 UTC · Comment
New Feature, theSkyNet Challenges
17 Feb 2014, 2:57:14 UTC
The current POGS challenge is flying along with SETI.USA way out in the lead. Pushing theSkyNet POGS up to a total of 62.76 TFLOPS! Amazing!
I've finally got a prototype of theSkyNet challenge system up and running. Incorporating the feedback I received from users in December. This week and next week I’ll be trialling it alongside the 2 running and upcoming challenges on BOINC Stats. Then after that I will open it up for everyone to create and join challenges.
I’m testing out too new challenge type, Peak RAC and total credit handicapped by number of users.
You can see the new challenges feature at http://www.theskynet.org/challenges
Planned but not yet implemented features:
It is important that this new feature is actually useful to our members so I will appreciate any and all feedback you guys have.
17 Feb 2014, 2:57:14 UTC
New work units
Over the weekend we'll stop giving out work units. One Monday I'll do some maintenance to the server and hopefully start processing a new batch of GALEX/SDSS/WISE data to test the new WISE data we have. 16 Jan 2014, 1:38:52 UTC · Comment
Happy Holidays from theSkyNet
We wish you all a happy holiday season, from everyone here at theSkyNet. To celebrate we've created a Christmas calendar for the 12 days leading up to Christmas. There's a new door to open every day with a stat/fact about theSkyNet, and some even have some extra goodies attached. It started on Friday, so there's already six doors for you to open. You can sign up to receive an email whenever a new door is unlocked for you. Enjoy!
We hope you and your families have a great holiday season, no matter what you're celebrating.
-TheSkyNet Team 18 Dec 2013, 1:28:10 UTC · Comment
Sorry about the outage one for the NFS disks died overnight.
The assimilator will be working hard all morning to catch up. 13 Dec 2013, 22:45:16 UTC · Comment
theSkyNet in Print
Its exciting news day.
Two of theSkyNet's scientists - Kevin Vinsen from ICRAR and David Thilker from Johns Hopkins University - have had a journal paper published talking all about theSkyNet and how it works.
Published in Astronomy & Computing the paper describes theSkyNet's technical side and how Kevin and David are using your donated processing power to calculate lots of different things about galaxies in the nearby Universe. It's a great summary of the science behind the beautiful galaxy images you're helping process in theSkyNet POGS.
You can see the paper's summary, and download it (for a fee) to read the whole thing, or read the archived pre-print version online.
But it doesn't stop there. We have also been featured as part of an AWS (Amazon Web Services) case study that explains how we utilise the AWS systems for efficiently managing POGS and theSkyNet website.
Enjoy! 12 Dec 2013, 8:33:36 UTC · Comment
Thank you to all our Challengers
We've had a great week for the Herschel's 275th Birthday challenge, thank you to all of our processors and the teams who participated. We made a new record - reaching over 50 TFLOPS in theSkyNet POGS last night. Thank you everyone!
Now, for some results:
The winning team of the challenge is SETI.USA, and the runners up were L'Alliance Francophone. With a total 88,301,765 credits delivered to theSkyNet: POGS users during the challenge, we can't believe how much crunching you guys have managed for us. We have gold and silver Herschel trophies for all SETI.USA and L'Alliance Francophone participants respectively, and a bronze for everyone else who helped out. These will be available in your trophy cabinet on theSkyNet.org soon.
As for our side challenge that we set yesterday - that was won by the amazing tanos, who had the highest RAC increase in the last 24 hours of the challenge. We'll be in touch for your address soon tanos so we can send you your prize.
An extra special thanks also needs to go out to Mumps [MM] who had the highest total credit contribution to the challenge, we'll also send you out a pack Mumps, so watch out for our message.
Thanks again everyone, you've helped us out immensely. We can't wait to go through the galaxy data that's come in this week and see what's been found.
Until next time,
TheSkyNet Team 22 Nov 2013, 6:17:18 UTC · Comment
Herschel Birthday Challenge 24 Hour Prize
Thanks to all our processors and those involved in the Herschel Birthday Challenge going on at the moment, theSkyNet POGS peaked yesterday at 49.22 TFLOPS! Thank you everyone.
There's only 1 days, 0 hours and 31 minutes to go until the challenge ends so to help us break through 50 TFLOPS of processing power and set a new record for the project we've decided to offer a bit of an incentive. The greatest increase in RAC in the next 24 hours (from the time of this notice) will receive a thank you prize from us, delivered anywhere in the world. It'll include a baseball cap, T-Shirt, USB drive and some other little items. Help us reach 50 TFLOPS today! 21 Nov 2013, 4:20:18 UTC · Comment
We reached 49.22 TFlops overnight
49.22 TFlops overnight
21 Nov 2013, 0:04:01 UTC · Comment
We've just gone over 48.5TFlops
20 Nov 2013, 9:10:24 UTC · Comment
Herschel's Birthday challenge update
Wow! Thanks to all those that are participating in the Herschel's Birthday challenge. We've reached almost 47 TFLOPS of processing power on theSkyNet POGS and over 51 TFLOPS on theSkyNet as a whole. Thank you so much for your help.
Over 700 galaxies have been processed during the challenge so far and we've got thousands more waiting, so we hope you keep at it - only two days left! Don't forget to check out theskynet.org in a few days for your special challenge trophy, we've got a gold version for the winning team, silver for the runner up and bronze for everyone else.
Our website developer is designing a new ‘Mini Challenges’ section for theskynet.org and would love your feedback on the forums.
Thanks again for joining us on theSkyNet POGS, your contribution is very much appreciated. 19 Nov 2013, 6:04:06 UTC · Comment
16 Nov 2013, 10:10:44 UTC
Thank you one and all.
We have records falling left, right & centre.
Looks like I'll need to do a big raid on the goodies cupboard :-). And yes we really do post goodies to people anywhere in the world when they win a prize.
16 Nov 2013, 10:10:44 UTC
CPU Usage for the first hour of the challenge
I love how everything went quiet 20 minutes before 5:00UTC. Then suddenly 7,000+ results were uploaded to be validated.
In the first hour over 5,000,000 credits were issued. Yesterday we gave out 6,205,611 credits.
15 Nov 2013, 6:35:53 UTC · Comment
Why is astronomy important
This very good article describes many of the things astronomy has contributed to our daily lives.
14 Nov 2013, 6:13:16 UTC · Comment
William Herschel - 275 anniversary of the birth - team challenge
In honour of William Herschel's 275th birthday, a BOINC challenge has been started on theSkyNet POGS by the Astronomy.Ru forum members! Check out all the teams participating here:
Welcome challengers - we're glad to have you crunching our galaxies for us. We're looking forward to seeing if we can set a TFLOPS record on theSkyNet POGS with your help! Don't forget to login (with your BOINC email and password) to www.theskynet.org and check out what trophies you accumulate over the week, you can also check out your contribution to theSkyNet with our graphs and stats system (and even see the galaxies you're helping us process.) We also have a special theSkyNet trophy for all the challenge participants, look out for it after the challenge ends next week.
Once the challenge is over we'll dig out some (small) prizes for random members of winning team and for the individual who processes the most. Your contribution to advancing our knowledge of the Universe is appreciated and will never be forgotten. No doubt future generations will sing songs about your generosity, foresight, courage and commitment to scientific endeavour. 14 Nov 2013, 6:02:51 UTC · Comment
Increased number of pixels
I'm trying an experiment over the weekend. I've increase the maximum number of pixels in a work unit to 28. I'm interested if it reduces the http load as we have an 18 team challenge starting soon 8 Nov 2013, 8:28:38 UTC · Comment
Check out the halloween theme on theSkyNet. 31 Oct 2013, 9:05:49 UTC · Comment
There will be a brief outage on Wednesday afternoon (Perth time) whilst I install the latest security updates on all the machines 27 Oct 2013, 23:35:32 UTC · Comment
Narrated slideshow describing theSkyNet POGS science background
Just wanted to remind everyone of the narrated slideshow we released during theSkyNet relaunch. I talk about the basic scientific premise of the work your computer is doing, show you what the PS1 telescope and camera really look like, explain the other space-based observatories involved, and give a preview of forthcoming goals.
Check it out at:
2 Oct 2013, 2:10:04 UTC · Comment
In more serious news, as of this morning (Perth Time), we are processing at 37.1TFlops a new record.
Thanks, Merci, Danke, Спасибо за поддержку, Arrr that be grand. 19 Sep 2013, 3:21:27 UTC · Comment
International Talk like a Pirate Day
Today is International Talk like a Pirate Day http://www.talklikeapirate.com.
TheSkyNet website offers a Pirate language option http://www.theskynet.org/?locale=arr to celebrate. 19 Sep 2013, 3:16:28 UTC · Comment
I've just queued another 819 PS1 only images of galaxies in NED with RA = 30-40 deg, DEC = -30-80 deg, and NED diameter of 1-3 arcmin. 16 Sep 2013, 2:09:35 UTC · Comment
TheSkyNet is celebrating its two year anniversary today with the official launch of a new web site, as well as a range of improvements and new features to make contributing to astronomical research at home more enjoyable, and even easier.
Go and have a look at http://www.theskynet.org. Your email (the one you registered when you joined BOINC) and password will get you in. All your BOINC credit is now part of theSkyNet and you may well have won some trophies.
A special thank you to all you BOINCers who helped Alex debug the site.
You can also see and hear David talk about the project here. He does a great job of explaining what we're doing. 13 Sep 2013, 0:50:45 UTC · Comment
A big thank you to the BOINC Australian who've also be doing the Australian Assault on POGS 12 Sep 2013, 23:05:17 UTC · Comment
Un grand merci à L'Alliance Francophone. Nous avons atteint avec le nouveau projet une meilleure performance! Nous traitons les données à une vitesse de 28.3TFlops, avec une cadence de 110 galaxies par jour. Nous espérons que theSkyNet POGS restera en permanence un de vos projets BOINC.
Kevin le «roast boeuf»
A big thank you to L'Alliance Francophone. We have reached a new 'project best' performance! We are processing at 28.3TFlops and processing 110 galaxies a day. We hope theSkyNet POGS stays as one of your permanent BOINC projects. 12 Sep 2013, 2:45:24 UTC · Comment
To the L'Alliance Francophone
Bonjour et bienvenue
To BOINC Australia
G'day - welcome back 9 Sep 2013, 3:35:48 UTC · Comment
We're currently processing PS1 only images of galaxies in NED with RA = 10-20 deg, DEC = -30-80 deg, and NED diameter of 1-3 arcmin.
I've just queued another 705 PS1 only images of galaxies in NED with RA = 20-30 deg, DEC = -30-80 deg, and NED diameter of 1-3 arcmin.
You folks are currently processing just over 100 galaxies a day. At this rate we'll finish the survey in 2-3 years. 9 Sep 2013, 1:08:24 UTC · Comment
Pan-STARRS1 Only Galaxies
We're now processing 511 Pan-STARRS1 (PS1) only images taken from the PS1 PV2 stacks. The galaxy selection is: all NED galaxies with RA = 0-10 deg, DEC = -30-80 deg, and NED diameter of 1-3 arcmin.
NED is the NASA/IPAC Extragalactic Database - you can find it here 16 Aug 2013, 1:31:23 UTC · Comment
I'm reprocessing the big galaxies to see what happens if I modify the sigma to 10% and then 5% 15 Aug 2013, 3:36:15 UTC · Comment
Android Message Board
For all you Android users I've created an Android specific message board 1 Aug 2013, 23:11:45 UTC · Comment
Area 2,000,000 has been released
Area 2,000,000 is somewhere inside M60. But there over 12,860 areas in that big boy 22 Jul 2013, 6:58:17 UTC · Comment
BIG Galaxy images
We've just put in some very BIG galaxies > 2000 x 2000 pixels. These are the biggest we've ever tried. We might have to do some quick changes to the web site to display these images.
It's 12,861 areas 22 Jul 2013, 5:14:05 UTC · Comment
New theSkyNet Web Site
18 Jul 2013, 7:36:09 UTC
After several months hard at work we are ready to begin testing the new webpage, theSkyNet2.0 (http://production-test.theskynet.org/).
I would like to extend the invitation to all of our amazing users, who have already been using the BOINC project, to join us in testing the new web site.
The new site is a complete rebuild of http://www.theskynet.org using Ruby on Rails.
We’ve retained all the existing features and added some exciting new ones. The most interesting I’ll list here:
• Boinc is now included! When you sign up you are giving the option creating either a nereus or a boinc account (or both). You total credit value is then a combination of the two systems
• New stats. I’ve added a graphing feature to the site that shows historical statistics dating back to the beginning of the project.
• Json API, this was a requested feature from the old site. You will find that most of the pages are capable of rending as a json document just append .json to the url to see what happens.
What does this mean for boinc users? Once signed in to the site you will be taken to your ‘Dashboard’ page (http://production-test.theskynet.org/profile) where you have access to:
• Your galaxies
• Statistics about your contributions to POGS
• Leader boards that bridge the gap between the original project and the POGS project
• theSkyNet.org alliance system
• And more
Anyone heading from POGS to the new site should be able to login with either your existing http://www.theskynet.org account (if you have one) or your POGS account details.
If you find any problems or have any feature requests could you please post them on the forums here under the website section (http://pogs.theskynet.org/pogs/forum_forum.php?id=11).
Thank you in advance for helping us test and improve the new theSkyNet webpage. This is a test site though and as we move out of testing in a few months’ time it is possible that accounts or alliances made on the site could be lost. Your BOINC credit is safe as I get it from the POGS system.
18 Jul 2013, 7:36:09 UTC
Our 2,000,000th area of the sky will soon be released. If you process it - I'll send you some goodies from ICRAR. 18 Jul 2013, 2:26:13 UTC · Comment
17 Jul 2013, 8:13:29 UTC
You'll notice a few changes, but a lot of the changes are behind the screens for this release:
In theory you shouldn't need to reconnect, but it is probably a good idea if you do. As you'll get a message like this everytime you connect:
You used the wrong URL for this project. When convenient, remove this project, then add http://pogs.theskynet.org/pogs/
Some people had to use the 'Reset Project' button in the boinc manager to make the link work properly.
17 Jul 2013, 8:13:29 UTC
We'll start the migration/optimisation on Monday. Hopefully the last 25 results will be in by then
This update will do the following:
12 Jul 2013, 9:26:32 UTC
No more WUs for week or two
I need to do some work on the database. So no new work units for a week or so as I need to get all those that are out there back in and then I can do the work I need to do. 1 Jul 2013, 1:18:18 UTC · Comment
Our first paper about this project is available for those that are interested at arXiv.org. This is just a detailed description of the project and the results to date.
We're working on a purely science focused one at the moment.
Depending on the budget available for next year (my AWS grant has run out) - we hope to:
17 Jun 2013, 3:24:45 UTC
Those of you who watch the stats might have notice one user is suddenly doing HUGE amounts of processing. Mark is testing a SGI UV2000 here at ICRAR. It is a single machine which gives use genuinely 256 cores and 2TB (yes that isn't a typo I do mean TerraBytes) of RAM. It also has a storage array with 50 x 300GB 10K drives plus 10 x 256GB SSDs - that is giving us about 1GB/sec write speed. When it isn't doing our Radio Astronomy number crunching tests we have it using its spare cycles on POGS.
They are cool machines - scalable to 2,048 cores and a whooping 64TB of RAM. You can add water cooling as an optional extra.
I love being a big kid with BIG TOYS. 22 May 2013, 1:26:45 UTC · Comment
The latest theSkyNet newsletter is here 13 May 2013, 2:16:47 UTC · Comment
Science result: Improved mass and star formation rate measurement from pixel-SED fitting
We wanted to share a plot showing some of the very early science results coming from theSkyNet POGS. There has been debate in the astronomy research community about the trustworthiness of estimated stellar mass and star formation rate values derived from spectral energy distribution (SED) fitting of entire galaxies at once. Traditionally, the total luminosity (at each wavelength) coming from the sum of all galactic sub-components is modeled without regard for the spatially-varying contribution of stellar populations having different characteristic age (old vs. young). This variation is clearly significant in certain galaxy types - for instance, it is one reason the spiral arms of a star-forming disk galaxy appear so prominently.
By lumping all the light from sub-components together before interpretation, potential biases are introduced into results from integrated (total) SED fitting. We have the opportunity to do better than this traditional approach. With your CPU-power, we can very efficiently fit for the diverse stellar populations seen on small (pixel-size) scales and only later sum our parameter maps (examples given in the earlier post linked below) across entire galaxies. In this way, we can use the extra information provided by the multi-wavelength morphology of each galaxy to get a more reliable answer for questions such as how much stellar mass resides in a galaxy, what is the rate of star formation, and how are mass/SFR related in galaxies of varied type.
Our POGS analysis for a small sample of objects run late last year shows that significant bias can be removed by working in the pixel-SED regime. In particular, using the traditional approach the stellar mass of a galaxy appears to be preferentially underestimated, by a factor which increases for decreasing galaxy mass. Also, a significant SFR bias is also demonstrated when comparing integrated-SED versus pixel-SED results. The plot we wanted to share (below) shows these results by drawing a vector for each galaxy in the mass-SFR parameter plane, connecting the estimated parameters evaluated via the two competing methods. The filled circles show the 'improved' location for each galaxy. This is not simply a trivial result. It likely has wide-reaching implications. As one example, scatter in the relation between stellar mass and SFR seen in this plot is reduced (the so-called galaxy main sequence becomes tighter), allowing us to track galaxy evolution with more fidelity. Adoption of the pixel-SED method will also influence interpretation of galaxy rotation curves and even the star formation law.
We are about to start using the much larger number of galaxies processed by now to attempt establishing parametric corrections for application to distant (progressively unresolved) galaxy populations. This would not be possible without your assistance -- thanks!
28 Mar 2013, 19:00:07 UTC · Comment
A word about our galaxy images
As volunteers on this project you may have noticed a wide variety in the telescope source and 'quality' of the galaxy images being processed. We think you deserve to know more about this.
Our project is rooted in the Pan-STARRS1 mission, a panoramic survey telescope now operating at the top of the Hawaiian island of Maui (on Mt. Haleakala). This innovative telescope uses the world largest digital camera (1.4 billion pixels) to repeatedly take photos of the sky through five filters ranging in color over the visible to near-infrared part of the spectrum. The tremendous speed with which PS1 can cover the sky enables it to find objects which move or change in brightness (e.g. you may have heard of Comet Pan-STARRS...) by looking repeatedly to the same areas. It also allows us to combine all these multiple exposures into one 'deeper', more sensitive view. The later approach of effectively staring is the one taken by our extragalactic project, though PS1 scientists have found time-variable things like cosmologically distant supernovae and super-massive black hole flares. The final, summed up PS1 images will have substantially more sensitivity and better resolution than even the world-famous Sloan Digital Sky Survey (SDSS), over a sky area much wider than imaged by SDSS.
However, as I said above, the PS1 telescope is still operating. The mission is now 75% complete, and we hope to continue observing until January 2014. The fact that the digital camera is built as a checkerboard mosaic of >3800 CCD separate detectors means that single (even multiple) exposures of the sky leave gaps on the sky. These are deliberately filled in over time as each part of the sky is observed, up to a maximum of 12 times per filter. The PS1 data now being processed (in our latest BOINC run) originates from a period when the PS1 survey was only about 60% finished, so gaps do remain. If you look at some of the multi-color galaxy data you've worked on (using the main page 'Images you have processed' link), it is not hard to find odd looking, frequently stripe-like regions of missing data in one filter or more. Rest assured that these will be long gone when the last photons are collected by PS1. As an example of what the eventual images will look like, check out our Astronomy Photo of the Day from earlier this year: http://apod.nasa.gov/apod/ap121012.html
So, you may wonder why we are processing the interim data from PS1. The answer is simple. We need to prepare our distributed-computing analysis system now, in advance. This means both the BOINC related and database management aspects (heroically headed by Kevin V. and team), plus the imaging and astrophysical interpretation components (to which the entire international team contributes). An illustration is understanding the degree to which our model parameters describing local galactic environmental conditions like regional stellar mass and star formation rate can be constrained with the available filters. This is one reason we also processed a large set of galaxies observed by SDSS. That project included a filter modestly bluer than the PS1 spectral window. It turns out this filter allows extra leverage in particularly active star-forming regions, dominated by hot young stellar populations. PS1 has the distinct edge in the red though, helping to quantify the spatial arrangement of old stars and peer through obscuring interstellar dust.
Our eventual goal is to combine image data reaching even further across the spectrum, adding ultraviolet light and infrared radiation detected by the space-based GALEX and WISE telescopes, respectively. We should be able to do this truly panchromatic analysis for about 30,000 galaxies large enough to be shown in adequate detail by the lower-resolution UV and IR data. Soon we will be conducting a test run of this GALEX-PS1/SDSS-WISE type for a small set of galaxies. As the PS1 survey is finished, we will ask you to crunch through our truly panchromatic data and process an entire sky full of a few-hundred thousand smaller, yet still resolved galaxies with visible light data only. The upcoming months will also witness some experiments with improvements such as: enhanced data preparation (spatially adaptive smoothing), hopefully GPU processing, plus consistency checks against a fully independent, second spectral energy distribution modeling code.
As ever, thanks for your help with our project! We could not do this without you! Even with the continually improving quality of PS1 imaging, we hope you enjoy looking at the targets you process. You may truly be one of the first people to ever inspect a digital color image for some of these galaxies (particularly if they occupy the southern sky).
22 Mar 2013, 10:00:04 UTC · Comment
March equinox challenge
Welcome to the participants of the March equinox challenge.
I've 2061 galaxies waiting for you to process.
20 Mar 2013, 9:56:56 UTC · Comment
Another Security Patch
Another security patch has been released so I'll be installing it this afternoon (Western Australian Time).
Should only cause a brief outage. 15 Mar 2013, 4:08:33 UTC · Comment
I'm putting the latest security patches from the BOINC trunk onto the server.
A fix for some serious security vulnerabilities in the BOINC server-side scheduler has been released 11 Mar 2013, 1:39:19 UTC · Comment
I've been archive data from the AWS database (which costs quite a bit as it is 1TB) to the iVEC Peta-Store. Then I delete data from the database that I don't need. I've inadvertently removed the link between the areas and users.
Fortunately it is all archived in the HDF5 files, so over the next couple of days I'll be retrieving the files from the Peta-Store and put the records back.
Sorry about that
1 Mar 2013, 3:37:27 UTC · Comment
We're processed 3,446 galaxies
These have generated 1,202,114 areas (squares on the galaxies) containing 19,798,628 pixels.
These pixels have generated 286,931,590 results (16 different results per pixel) with a total of 6,174,230,796 probability distribution points.
Thank you everyone 26 Feb 2013, 8:50:39 UTC · Comment
2924 Galaxies in the queue
And we're back. We've put 2924 galaxies in the queue for you to process 26 Feb 2013, 4:43:27 UTC · Comment
No more WUs for a few days
I need to do some work on the database to recover space. I'm going to turn off the work unit generation for a few days to stop the flow of data. Then we have another 2,000 PS1 galaxies to process 21 Feb 2013, 3:26:03 UTC · Comment
More details on the disk crash
The CPU stats for the last week are shown in the graph before
The blue line (i-06bab47e) is the public facing server and the green line (i-b89e41c7) is the second server.
The main server runs the web site, upload & download, validator and assimilators. The second server runs the tasks - which includes keep the queues feed.
Until the 25th we were stressing the second server, but not the main one. Once the challenge started both servers started to get loaded. They survived for a couple of days; then on Sunday morning the disk on the main server crashed up under the load. Fortunately I'd just gotten up to feed the horses (at 5am) and noticed it.
I stopped the servers and moved them from the m1.small to the c1.medium instance type under AWS and restarted them.
As you can see the load on the main server is now quite manageable, whilst the second server is peaking when I have to load the BOINC queues with more galaxies. 29 Jan 2013, 3:26:23 UTC · Comment
Area 1,000,000 is on the queue
Area 1,000,000 has been generated and is on the queue. NGC3032_area1000000.
When it is issued to the crunchers and the answer returned - I'll contact the lucky winners directly 28 Jan 2013, 8:07:53 UTC · Comment
Well the load got to the system and crashed the disk on the main server. I use GlusterFS to replicate the disk live, just in case - so I was able to recover pretty much everything. I'll be sending the logs to the GlusterFS community so they can see if they can see what happened. It looks to be like the load just got too much and the FS started to miss blocks.
Whilst the disk rebuilt, I upgrade the two server from an AWS m1.small instance to a c1.medium so I can cope with more load (I hope).
Thanks for you patience. 27 Jan 2013, 9:23:14 UTC · Comment
Within the next couple of days (probably early Monday Western Australia time), two of you crunchers will process area 1,000,000
For the two people who process it - I'll send you some ICRAR astronomy goodies as a prize. 26 Jan 2013, 10:48:04 UTC · Comment
I've had to add another 200GB to the database.
Such a shame I set this DB up before Amazon introduced the guaranteed IOPS. I could really use it now. The only way to get it is to recreate the database from scratch and then copy from one database to the other (which for 800GB might take a day or so).
I'll have a look at doing this after the challenge has finished. 26 Jan 2013, 9:41:05 UTC · Comment
theSkyNet POGS team welcome the challenge crunchers.
You guys are awesome. We're generating more galaxies for you to Crunch over the weekend. Yesterday I had to ramp up the generator as you are sucking down 3 galaxies every 10 minutes. Thats 432 a day! In the past 6 months we'd only processed 2,300 galaxies. You'll do more than that in a week!
The BOINC servers are just about handling the load, but the Database has been struggling. I might have to up it to the next size Amazon Instance
25 Jan 2013, 23:21:11 UTC · Comment
Challenge is consuming galaxies too quickly
At the currently rate of WU consumption you will process all 1,800 remaining galaxies by next Tuesday. Basically you've quadrupled the throughput.
Unfortunately I don't have any more prepared. So we might have to redo some of the old data. 25 Jan 2013, 8:35:24 UTC · Comment
Australia Day Weekend
It's Australia Day tomorrow and therefore a long holiday here in sunny Western Australia.
I'll be keeping an eye on the system, but not 24x7. If there are continuing DB issues it may take me a few hours to fix them. 25 Jan 2013, 8:14:56 UTC · Comment
I've just spun up a couple more assimilators to keep up with the challenge 25 Jan 2013, 7:07:14 UTC · Comment
I'm having some issues with the Database in the cloud at the moment. Waiting on Amazon to resolve.
25 Jan 2013, 4:49:55 UTC · Comment
Data processed so far
18 Jan 2013, 3:03:32 UTC
18 Jan 2013, 3:03:32 UTC
2,000+ Galaxies still in the pipe
There are still 2000+ galaxies in the pipeline 2 Jan 2013, 22:47:09 UTC · Comment
We lost the Gluster Filesystem for about 4 hours this afternoon
Hopeful it is all better now 19 Dec 2012, 8:39:59 UTC · Comment
Short outage this morning
There will be a short outage this morning whilst I add another 200GB to the database 10 Dec 2012, 1:25:18 UTC · Comment
We've just gone past 500,000 areas
Thanks everyone 7 Dec 2012, 8:40:24 UTC · Comment
theSkyNet POGS welcomes the Aussie Assault. 2 Dec 2012, 22:40:30 UTC · Comment
100,000,000 Credits issued
27 Nov 2012, 5:12:59 UTC
Thank you everyone - overnight we passed 100,000,000 credits issued according to http://boincstats.com/en/stats/projectStatsInfo
So far you've crunched:
David and I really appreciate your participation
27 Nov 2012, 5:12:59 UTC
Might run out of galaxies overnight
We might run out of galaxies overnight (Western Australia night).
The next batch should be ready in a day or two. 26 Nov 2012, 7:48:56 UTC · Comment
We are starting a new run (suffix 'f') of the same test galaxies featured in the early science post. This time we're including SDSS-u imaging as a complement to the PS1 grizy spectral coverage. A similar run was done earlier, but now we're incorporating realistic uncertainties for all filters. The new run will tell us how accurately we can constrain our derived physical parameters using PS1 alone for high-resolution runs. After this test we will finally work on some other galaxies -- having finished important checks with the limited sample.
We're also now running on a dual server configuration in the AWS cloud. This will give us some measure of how we can scale out this project as more users come on-board 22 Nov 2012, 3:39:04 UTC · Comment
Exciting early science results from all your work!
In our last test run (galaxies given the 'e' suffix), we analyzed galaxies in a small section of the sky already having the equivalent of 3-year PS1 coverage. One of the goals was to explore the information content of the pixel spectral energy distributions (pixel-SEDs) containing exclusively PS1 data and at ~1 arcsec resolution. Your volunteer computers processed several hundred small [~1 arcmin diameter] galaxies using only PS1 "grizy" pixel-SEDs (not yet including GALEX and WISE). The "grizy" filter set spans wavelengths from 480 nm (g-band) to 1 micron (y-band). The first figure shows PS1 z,r,g color composites (left), stellar mass surface density (total mass of all stars existing per unit area, center) and star formation rate surface density (stellar mass formed per year per unit area averaged over the past 100 Myr, right), for six example galaxies. The results are quite encouraging, and will be improved even further with true panchromatic spectral coverage from GALEX UV to WISE IR. As expected, features tracing recent star formation (such as spiral arms) are much less obvious in the stellar mass maps. Galactic bars are (linear structures centered on the nucleus, such as in the second galaxy on the left) are quite nicely defined in the stellar mass images when present. In true production runs foreground stars will be masked. The lower-left galaxy intentionally shows some pixels not yet returned by volunteers at the time the figure was created, to illustrate the chunks processed by each person.
The second and third figures show the majority of galaxies processed in test run 'e', displaying the stellar mass (color) and SFR surface density (greyscale) respectively. In total there were nearly 0.5 million pixels processed. This would have taken about 10 years locally on a single typical core, but was finished in only a few days thanks to your help!
A research paper is currently being written about these results, for submission to one of the professional astronomy journals.
Thanks again! More data coming soon... 21 Nov 2012, 3:49:59 UTC · Comment
Twin Server Configuration
We're gearing up for the next big release so we now have two servers in our BOINC cluster and I'm just about to start configuring the MongoDB components
The images so far are very good; the mean squared error between the 10% noise and the real noise was enough to make it worth while us doing it more future runs if we can get the data. 15 Nov 2012, 8:40:56 UTC · Comment
About to run out of WUs
We're about to run out of data.
David and I need to analysis the results of the last three runs as this will influence the pipeline for the full run. So it may be a few days before we get more data for you to crunch 6 Nov 2012, 21:53:44 UTC · Comment
200+ new runs on old galaxies
We're doing the "c run" SAS galaxies again but with actual uncertainties (we'd guessed at 10% in previous runs). So now we have the signal-to-noise for each pixel per filter with all S/N values below 1 forced to equal 1 (so as not to blow up the inferred noise in such pixels).
These have the postfix "e" 2 Nov 2012, 5:44:02 UTC · Comment
We hope to release more galaxies tomorrow 1 Nov 2012, 1:01:29 UTC · Comment
We now have so many users I've had to upgrade the AWS Database instance from small to large. I'm glad I have a grant for this....
We'll be switching to a MongoDB instance (when I finish writing the code) as that has a much higher ingest rate. 25 Oct 2012, 8:46:32 UTC · Comment
The assimilators have fallen quite a way behind because Amazon had some major IO issues yesterday - we should catch up in a day or so 24 Oct 2012, 1:25:41 UTC · Comment
450 new galaxies
We're looking at many different redshifts this time and comparing Pure PanSTARS images (the galaxies ending with c) with SDSSu + PanSTARRS images (the galaxies ending with d). 22 Oct 2012, 1:53:05 UTC · Comment
No more galaxies for a couple of days
The database is now 200GB and I need to do some housekeeping to extend it for the next run. Should be back again by Wednesday or Thursday. 13 Oct 2012, 22:59:58 UTC · Comment
We've just put more data in the queue.
We're using the same galaxies, but with the Sloane SDSSu band added. The PanSTARSS1 data has been convolved to the resolution of SDSS u (~1.3"). We want to do a comparison run with the PanSTARRS1 convolution. Then we can compare directly PS1 to PS1+SDSSu.
11 Oct 2012, 1:42:28 UTC · Comment
We're out of WUs
But more images will arrive in the next day or so. 9 Oct 2012, 7:28:42 UTC · Comment
Holy super number crunchers Batman
At this rate you'll have taken all the WU for this run by tomorrow! 8 Oct 2012, 0:47:06 UTC · Comment
Very fast WUs
I wish it was because of my super-duper optimisation skills, but it's not. Basically I "cocked up".
The F77 code looks for model files within a z (redshift) of 0.005. I misread this as 0.05. What is happening is the F77 code just says - "I can't do this" and exits nicely.
As one of the models is at z 0.05 and 107 galaxies are at this z I've moved them to the front of the queue.
I'll spent 10 minutes in corner with the "Stupid Kevin" hat on as penance - then I'll fix it. 6 Oct 2012, 23:58:38 UTC · Comment
The new version needs new models, unfortunately the BOINC client will start trying to run things before it has finished downloading them.
It'll either go really quickly or fail with computation errors. I'll register the early galaxies again so they get processed. 6 Oct 2012, 4:09:21 UTC · Comment
5 Oct 2012, 9:30:42 UTC
We're ready to go with the new galaxies, we're just waiting for the last few results of the old run to come in. We've made a number of improvements:
1) The F77 records more probability data than before;
2) We've decided to push the redshift out to 0.1;
3) The next run is pure PanSTARRS1 data;
4) The 64bit Windows code should be a lot faster;
5) A improvement to the Cobblestone scaling to look at the number of layers you are processing as well as the number of pixels.
WARNING: The model files have grown in size so you'll get a once of down load of 20MB (gzipped)
5 Oct 2012, 9:30:42 UTC
Maintenance of Database
Firstly thanks for your help so far. The BOINC community are truly wonderful. David and will be working on a couple of papers over the next few months (and yes, when they are published we'll post links so you can see what we've been doing with your CPU cycles).
After the new sigma run is finished - we'll not be issuing any new work units for a couple of days as I need to massage the database which is sitting at just under 200GB. I also need to modify the F77 to store the highest probability bin values - which means the new code will not validate against the old stuff.
Once I've do that we'll have a test with the first "real" hi-res images from PanSTARRS to make sure all is working correctly - then we'll run a couple of hundred local galaxies through the system. 2 Oct 2012, 9:57:54 UTC · Comment
We're going to reprocess some of the galaxies with a smaller sigma (uncertainty) to see the effects on the chi-squared value.
So when you start seeing galaxies with an 'a' postfix - that is what they are.
1 Oct 2012, 3:03:43 UTC · Comment
New WUs - fixing convergence problem
From NGC4612 to NGC7782 and for the PGC and UGC galaxies I've fixed the work units so we shouldn't see the convergence problem.
This means we should see the return to normal length work units as the fitting will converge to the correct probability bins.
25 Sep 2012, 1:26:40 UTC · Comment
Links to telescope and software pages added
On the main page we have added image map links to the POGS (PS1 Optical Galaxy Survey) project banner, allowing those who are interested easily navigate to the websites for GALEX, PS1, WISE, and MAGPHYS. GALEX (the Galaxy Evolution Explorer) is a space telescope operating in the ultraviolet spectral range, which is an excellent tracer of recent star formation sites. PS1 (Pan-STARRS1) is an optical survey telescope conducting observations from Mt. Haleakala on Maui. It uses the world's largest digital camera (1.4 Gigapixels!) to repeatedly image the entire northern sky. WISE (the Wide-field Infrared Survey Explorer) is another small NASA mission that recently completed all-sky imaging in multiple infrared bands sensitive to old stars, the interstellar medium, and warm dust in galaxies. MAGPHYS is the spectral energy distribution (SED) fitting code you have all been running via BOINC, allowing us to estimate physical parameters based on the multi-wavelength data from these three complementary telescopes. 22 Sep 2012, 17:15:00 UTC · Comment
Drip feeder about to kick in
The drip feeder will kick in later today.
I've got 64,000 WU waiting in the wings. 21 Sep 2012, 7:42:22 UTC · Comment
As space is getting a bit tight in the database - I'm deleting the older galaxies from the early part of the testing. 21 Sep 2012, 7:22:22 UTC · Comment
I've modified the F77 code to ignore pixel combinations that won't converge.
Hopefully (fingers and toes crossed) this should stop the invalid tasks we've been seeing lately.
I also feel confident enough to start turning on the optimisation so hopefully it will crunch quicker. 19 Sep 2012, 7:42:16 UTC · Comment
I've turned off Homogenous Redundancy for a bit to see if we get the same types of errors I'm seeing with HR on.
The upshot is all those pending revalidation jobs now move to the front of the queue.
17 Sep 2012, 7:59:13 UTC · Comment
The load test will soon enter a new phase. You may have noticed in the galaxy list a lot of the galaxies have 0% complete. This is because they are waiting in the wings.
Once the number of jobs available drops to 15,000 I'll start releasing whole galaxies so I can get around the way Homogenous Redundancy schedules things. Hopefully this will get jobs rescheduled much quicker.
16 Sep 2012, 2:54:08 UTC · Comment
I've turned on matchmaking - hopefully those of you with the "venerable" XP machines should start getting smaller WU's, whilst those with the BIG grunt boxes will get the bigger ones. 3 Sep 2012, 5:31:34 UTC · Comment
David's presentation on POGS
Here is a copy of David's presentation to the PS1SC - it's 11MB (lots of pictures) 3 Sep 2012, 2:49:16 UTC · Comment
Display all Galaxies
A new option has been added to the front page to display images of all the galaxies that are used in this survey. This is an initial version and will be improved. We are looking for suggestions on best way to organize the galaxies to make this more useful. Any comments are appreciated. 31 Aug 2012, 7:44:36 UTC · Comment
Load Test Status
WU available - 138,941
WU In Progress - 6,985
WU In Reserve - 59,592 - I've written a trickle feeder (for later)
CPU - 71% idle
Swap used - 0
DB Size 32GB (we store a lot) and growing
Users - 2121 (Not bad for a Beta test -thanks)
30 Aug 2012, 4:49:13 UTC · Comment
Load test time
Right folks - load test time.
David has been busy at Johns Hopkins University and made some modifications to his code for making the POGS image files, allowing substantially more pixels to be considered for a fit (now each band is masked independently). In this iteration (v3) he has used constant surface brightness limits across the sample with the exact value dependent upon band (28.0, 28.0, 26.0, 27.2, 27.1, 26.0, 25.0, 22.6, 21.0, 18.0, 16.2 ABmag/sq.arcsec for FUV, NUV, u, g, r, i, z, w1, w2, w3, w4).
We've released 188 galaxies for this load test.
Hopefully on Friday we'll have the "Lite" version ready for people to process 5 - 10 pixels a go on the more venerable systems out there.
I've also added limits to try and stop people sucking too many units 29 Aug 2012, 6:58:30 UTC · Comment
We found a mistake in the co-ordinate system - which we've fixed (I hope) so we'll redo 9 galaxies to test 24 Aug 2012, 7:06:59 UTC · Comment
The mail is now flowing again 24 Aug 2012, 7:06:10 UTC · Comment
I've not put any more galaxies in the pipe as we need to check the scaling of the redshift is working properly.
We should have more data in a week or so. The only data you'll see is rework jobs or revalidation 19 Aug 2012, 22:44:47 UTC · Comment
Rob has been busy - check out the new images we are generating on the fly as results come in.
mu = fraction of the total V-band optical depth of the dust seen by young stars contributed by dust in the ambient interstellar medium
ldust = luminosity absorbed and reradiated by dust
m = effective stellar mass
sfr = star formation rate 17 Aug 2012, 5:11:05 UTC · Comment
Prize for area10000
The 2 people who correctly process area10000 will win a small prize.
I'll send you some ICRAR goodies to where ever you live in the world. 17 Aug 2012, 4:29:26 UTC · Comment
Wow!! I release 11 galaxies and all the WUs are taken in 40 minutes.
I've put out a lot more smaller ones to. 16 Aug 2012, 12:45:48 UTC · Comment
We're now going to do some tests on scaling the redshift. We don't want you to have to download a 17MB model for every galaxy so we're going to try it with 3 models and scale the results.
We'll be looking at:
16 Aug 2012, 6:06:56 UTC · Comment
Thanks for you help on this so far. We're now looking at the results to make sure they make sense before we put out more galaxies.
We've got 216 ready, but I want to make sure you donate your spare-cycles to good science. 13 Aug 2012, 2:38:43 UTC · Comment
On the front page you should see a link that will allow you to see images of the galaxies you have worked on and where you contributed. 10 Aug 2012, 3:22:27 UTC · Comment
I've ditched sendmail and dropped postfix in its place. Wonder of wonders the mail is starting to flow. 9 Aug 2012, 8:01:36 UTC · Comment
I've moved the server to a slightly bigger instance in AWS and opened up the project again.
Feel free to invite your friends
9 Aug 2012, 2:27:20 UTC · Comment
Some very good news. The results we have so far will be presented at the next PS1SC meeting.
Once the images are pretty enough I'll put them up for you to see where you contributed. 9 Aug 2012, 2:21:53 UTC · Comment
New Work units
We found one of the filters was incorrect so I've released all the galaxies as new work units.
The new work units have my scaling factor in to make things more comparable with Cobblestones.
You can ditch any existing work units you are processing with a "Project Reset" to make sure you have all the correct models loaded.
9 Aug 2012, 2:15:24 UTC · Comment
7 Aug 2012, 9:07:55 UTC
I've fixed a few of the problems you folks have identified.
The F77 does not produce numbers to as many decimal places
The validator is more flexible and should now be happy with *Nix vs Windows generated numbers
I've supplemented the checkpointing with my own so we can really see what is happening
Moved up to the m1.small AWS instance
The weird time estimates on the wrapper is proving a thorny one.
7 Aug 2012, 9:07:55 UTC
88 Testers - Awesome
I now have 88 testers - you guys are awesome. I'm really surprised that my t1.micro instance on AWS (the free one) is coping nicely - peak CPU load is only 30%. The BOINC developers have down a fantastic job of minimising the CPU requirements on the servers.
I've closed account creation now - as I only have 6 galaxies prepared and we're already 1/2 way through it in 3 days!!! I'll speak to my colleagues at Johns Hopkins to see if we can get some more as validation does not appear consistent Windows & OS X seem ok, but Linux appears variable. 5 Aug 2012, 0:34:56 UTC · Comment
Ladies and Gentlemen
I have a simple question. Is there anything I, as the science geek, could provide that would make this type of project more appealing. My first thought was showing you on the real (false colour) images of the galaxies you've been processing, with the bits you actually did shown and what we found out.
Other ideas are to link it into google sky http://www.google.com/sky/ so you can see where in the sky that particular galaxy is.
Anything else you can think of? 4 Aug 2012, 8:23:57 UTC · Comment
Welcome and thank you for helping me test the next theSkyNet application.
We now have 27 users which should be enough for this run. 3 Aug 2012, 9:02:15 UTC · Comment