Monday, February 22, 2016

Ham Radio: Digital Mode JT-65 and JT-9

Amateur Radio is another one of my hobbies.  In January I found this new Digital Mode called JT-65, so far I have used it on 10 meters through 40 meters and listened on 6 meters and 80 meters.  What is really neat about this mode is less power is more.  10 watts on 20 meters I worked Reunion Island from Longview, Washington and based on the signal report I could have probably used even less.


I won't go into what JT-65 is, but since you can read tons about it here.  WSJT stands for "Weak Signal Communication by K1JT."  The mode is used for EME (Earth-Moon-Earth) communications, meteor scatter and HF/LF communications.  
The JT-65 QSO is very short but very long.  You can only send 13 characters, uppercase A-Z, 0-9, slash(/), hyphen(-) and the space character.  Each side of the QSO takes 1 minute.  Yes! 1 minute so a standard QSO, from QSO to sign-off is 6 minutes.  Typically you will see something like:

CQ N7DQ CN86             <-- basic call for a QSO with grid square
N7DQ KG7EYC CN86    <-- Please talk to me
KG7EYC N7DQ -01        <-- Signal report for KG7EYC from N7DQ
N7DQ KG7EYC R-01      <-- Acknowledge signal report and send a signal report
KG7EYC N7DQ 73          <-- Thank you for the QSO
N7DQ KG7EYC 73          <-- Thank you for the QSO 

Remember this is one minute for each phase of the QSO, well actually, 47 seconds sending the transmission and 13 seconds of listening, deciding what to do next. Look at the list of stations, decide which one you want to talk with, and click on the call and ensure you have transmit enabled.

You can hear JT-65 on multiple HF and even VHF frequencies.  The sound reminds me of music from a child's music box.  This link provides a list of those frequencies and recommendations on Good Operating Procedures.  

Now your are thinking how do I try this mode out.  First I am assuming you already are doing some form of digital communication from your computer through your radio such as RTTY or PSK31.  If not, you will want to get a link between your computer and radio using something like a SignaLink USB or checkout and see if your radio can connect directly to your computer via a cable and do digital modes.  You should need a simple cable that you should be able to purchase from your radio manufacture or the place you bought the radio.

One other connection you might want to setup is Rig Control, this way you can use your computer to control your radio.  Again, the cable should be available via your radio manufacture or your radio vendor.  Search on Google for your radio and rig control or CAT, for example,  "Yaesu FT-897 CAT" or "Kenwood TS-850 Rig Control" and you should find something to point you in the right direction.

Now for the process to set things up. Your computer clock must be accurate to within a second or two.  From my experience with Windows 7, 8, and 10, Windows only updates the clock about once a day and for some laptops the time drift can be big.  First download Dimension4.  It is a basic install, accept the defaults, once installed start it up and configure it to start on Windows startup.  I have found that if you put your computer to sleep, having this running will help with time drift issues.

Next, download WSJT-x (1.6). During the install it will ask some basic questions.  There is a question about using hamspots.net, I recommend you take the default. While you run WSJT-x, it will report what stations you hear and the signal report you would give them.  You can later change this to pskreporter.info if you want and if you want to see a map of connection  go here. What is cool about this is you will see how far your signal can go around the world.








Image from hamspots.net spots:



Image from pskreporter.info spots on a map:


Finally, download JT Alert  - make sure you get the supporting applications as well. Install in the order downloaded.  JT Alert provides an interface to some logs and helps you track needed contacts for state, country or grid.

Basic quick start:
Start WSJT-x, go to File->Settings. Fill out the first tab with your information.  




Next if you have a rig control cable, go to the rig tab and configure it for your radio, 



next go to the audio tab and setup the connection to your sound card interface. 



There are some other options that are beyond this simple article and you will some good setup directions out there.  Read some of the docs for WSJT-x on setting things up, basically about 15 minutes, maybe 30 minutes or if you don't skim an hour.  This mode and setup is easy, follow the quick guides and get on the air.

Now make sure you started Dimension4, ensure it syncs your computer clock and is set to startup on booting the computer. Start WSJT-x followed by JT Alert. Set your radio to the recommended frequencies in WSJT-x and listen, let the program decode.

Focus on JT-65 not JT-9 at this time. Make sure you set your radio output power to no more than 10 watts, yes that is right 10 watts. More than that you will be splattering the band with noise and causing interference for other stations.  If you are getting lots of -01 reports, you might consider decreasing your power.  There is a site that will help you do a signal report and power comparison called dBCalc: Signal Power & dB Comparison.  One other reason to keep the power low is that this mode is full key down during the entire communication.  If you are using more than 50 watts or 100 watts, you could have some issues with your finals life span.

What I find interesting and cool is how well, WSJT-x pulls signals from the noise, here is an image of a station calling from Japan and for me is right in some noise either caused by propagation conditions, antenna issues, computer noise, radio noise or interference.  You can even have signals that are crossed over each other see both QSOs. Though if two transmitting signals are exactly on each other you may not decode either and the music is not good.

Go out to Youtube and do a search for WJST-x and find a couple of videos about operating in this mode -- probably should have told you this first but we are amateur radio operators and we try stuff before reading everything.

There are several other programs that do JT-65/JT-9 but I have found that I like the way WSJT-x works by keeping things simple.  I have downloaded and used the programs and they have some pretty neat features as well.  Here is a list of the other programs.  JT Alert will interface to each of them as well, so you may need to reinstall JT Alert to configure it to see these programs.

JT65-HF-Comfort by DL3VCO

JT65-HF HB9HQX Edition

JT65-HF by W6CQZ looks like it is no longer available and the project is abandoned.

Hope you give this mode a try and you should see me out on the air at some point.

73 de Mike - N7DQ

Saturday, August 29, 2015

Perl and Flickr Stats and Google Docs - Updated for Google OAuth2

In this update article of Perl and Flickr Stats and Google Docs, I'll be covering how to change the final program to use Google OAuth2 for accessing the Google Spreadsheet. Please review the previous posts, especially number 4 in the series as I'll be referencing that code to add in specific changes for OAuth2.

The reason for this update is that some time back around May and June of 2015, Google changed the requirement in using their APIs from just username/password to OAuth2.  I did not notice this until recently and finally got time to update the access to Google Docs.  The one thing I need to add to this program is some error checking so it can email me if an error occurs and I can deal with future issues sooner.

The previous posts cover connecting to Flickr, pulling the data, authenticating to Google Docs and putting the data into the spreadsheet; Perl and Flickr Stats and Google DocsPerl and Flickr Stats and Google Docs - Part 2Perl and Flickr Stats and Google Docs - Part 3 and Perl and Flickr Stats and Google Docs - Updated for Google OAuth2.  You will want to review the previous articles as they fully document what is happening and the setup for Google OAuth2 and changes to the main program.

The setup for Google OAuth2 is almost like what you would do for the Flickr API Auth.  In this case you need to get an account on developers.google.com, Once in, create an OAuth2 ClientID, I selected other rather than Web Application, and you will get a ClientID and a client secret to use in the program to get and save the authorization code. The base code I used was from http://stackoverflow.com/questions/30735920/authenticating-in-a-google-sheets-application and http://www.stackoverflow.dluat.com/questions/31527911/perl-using-netgooglespreadsheets-gives-error-in-libxml.

Enter this code, update your Client ID and client secert, save the program and run it.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
#!/usr/bin/perl

# Code to get a web-based token that can be stored
# and used later to authorize our spreadsheet access.

# Based on code from https://gist.github.com/hexaddikt/6738162

#-------------------------------------------------------------------
# To use this code: 
# 1. Edit the lines below to put in your own 
#    client_id and client_secret from Google.  
# 2. Run this script and follow the directions on
#    the screen, which will give step you
#    through the following steps:
# 3. Copy the URL printed out, and paste
#    the URL in a browser to load the page.
# 4. On the resulting page, click OK (possibly
#    after being asked to log in to your Google
#    account).
# 5. You will be redirected to a page that provides
#    a code that you should copy and paste back into the
#    terminal window, so this script can exchange it for
#    an access token from Google, and store the token.  
#    That will be the the token the other spreadsheet access
#    code can use.


use Net::Google::DataAPI::Auth::OAuth2;
use Net::Google::Spreadsheets;
use Storable; #to save and restore token for future use
use Term::Prompt;

# Provide the filename in which we will store the access
# token.  This file will also need to be readable by the
# other script that accesses the spreadsheet and parses
# the contents.

my $session_filename = ".stored_google_access.session";


# Code for accessing your Google account.  The required client_id
# and client_secret can be found in your Google Developer's console
# page, as described in the detailed instruction document.  This
# block of code will also need to appear in the other script that
# accesses the spreadsheet.

# Be sure to edit the lines below to fill in your correct client
# id and client secret!
my $oauth2 = Net::Google::DataAPI::Auth::OAuth2->new(
client_id => 'CLIENT_ID.apps.googleusercontent.com',
client_secret => 'CLIENT_SECRET',
   scope => ['http://spreadsheets.google.com/feeds/'],
);
# We need to set these parameters this way in order to ensure
# that we get not only an access token, but also a refresh token
# that can be used to update it as needed.
my $url = $oauth2->authorize_url(access_type => 'offline',
approval_prompt => 'force');

# Give the user instructions on what to do:
print <<END

The following URL can be used to obtain an access token from
Google.  

1. Copy the URL and paste it into a browser.  

2.  You may be asked to log into your Google account if you
were not logged in already in that browser.  If so, go
ahead and log in to whatever account you want to have
access to the Google doc.

3. On the next page, click "Accept" when asked to grant access.

4.  You will then be redirected to a page with a box in the
left-hand column labeled  "Authorization code".  
Copy the code in that box and come back here.

Here is the URL to paste in your browser to get the code:

$url

END
;

# Here is where we get the code from the user:
my $code = prompt('x', 'Paste the code obtained at the above URL here: ', '', '');
 
 # Exchange the code for an access token:
 my $token = $oauth2->get_access_token($code) or die;

# If we get to here, it worked!  Report success:
print "nToken obtained successfully!n";
print "Here are the token contents (just FYI):nn";
print $token->to_string, "n";

# Save the token for future use:
my $session = $token->session_freeze;
store($session, $session_filename);

print <<END2

Token successfully stored in file $session_filename.
 
 Use that filename in your spreadsheet-access script to
 load the token as needed for access to the spreadsheet data.

END2
   ;

It will produce a URL that you will copy and paste into your browser, taking you to an access approval screen, which you will click Accept on, taking you to a page with a code, which you will copy and enter back to the waiting program and hit return.  It will save your authorization information into a file called .stored_google_access.session.  You will want to change permissions to 600 and move it to your $HOME directory in Linux so the cron job will have access to it or add the full path into the base program below

Here is the program flickr-google-stats.pl, I'll be walking through the code below.

flickr-google-stats.pl


  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
#!/usr/bin/perl
use strict;
use warnings;
use feature qw(say switch);
use POSIX qw(strftime);
# Note: the order of these three lines is critical
# the Net::Google::Spreadsheets comes after the Flickr
# modules you will get errors but you will get data
# in the spreadsheet
#
# Class::MOP::load_class is deprecated at /usr/local/lib64/perl5/Class/MOP.pm line 69.
# Class::MOP::load_class("Net::Google::Spreadsheets::Spreadsheet") called at /usr/local/share/perl5/Net/Google/DataAPI.pm line 108
# Net::Google::Spreadsheets::spreadsheets(Net::Google::Spreadsheets=HASH(0x22c6868), undef) called at /usr/local/share/perl5/Net/Google/Spreadsheets.pm line 57
# Class::MOP::Class:::around(CODE(0x3e90790), Net::Google::Spreadsheets=HASH(0x22c6868)) called at /usr/local/lib64/perl5/Class/MOP/Method/Wrapped.pm line 158
# Class::MOP::Method::Wrapped::__ANON__(Net::Google::Spreadsheets=HASH(0x22c6868)) called at /usr/local/lib64/perl5/Class/MOP/Method/Wrapped.pm line 87
# Net::Google::Spreadsheets::spreadsheets(Net::Google::Spreadsheets=HASH(0x22c6868)) called at google-input.pl line 16
#

use Net::Google::Spreadsheets;
use Net::Google::DataAPI::Auth::OAuth2;
use Net::OAuth2::AccessToken;
use Storable;

use Flickr::API2;
use Flickr::API2::Request;

my $username = 'Username';
my $apppassword = 'Password';

my $flickrKey = 'FLICKR_KEY';
my $flickrSecretKey = 'FLICKER SECRET KEY';
my $flickrAuthToken = 'FLICKER AUTH TOKEN';

# Ensure you format the Date Column in the spreadsheet for this format
my $date = strftime "%Y-%m-%d", localtime;

print $date . "\n";

my $api = new Flickr::API2({'key'    => $flickrKey,
                             'secret' => $flickrSecretKey,
                             'perms' => 'read'});

my $gtv = $api->execute_method("flickr.stats.gettotalviews",
         {date=>$date,
          auth_token=>$flickrAuthToken}); 

say "Total views is " . $gtv->{stats}->{total}->{views};
my $totalviews = $gtv->{stats}->{total}->{views};

say "Photostream views is " . $gtv->{stats}->{photostream}->{views};
my $photostreamviews = $gtv->{stats}->{photostream}->{views};

say "Photos views is " . $gtv->{stats}->{photos}->{views};
my $photoviews = $gtv->{stats}->{photos}->{views};

say "Sets views is " . $gtv->{stats}->{sets}->{views};
my $setsviews = $gtv->{stats}->{sets}->{views};

say "Collections views is " . $gtv->{stats}->{collections}->{views};
my $collectionsviews = $gtv->{stats}->{collections}->{views};

say "Galleries views is " . $gtv->{stats}->{galleries}->{views};
my $galleriesviews = $gtv->{stats}->{galleries}->{views};

 my $gpp = $api->execute_method("flickr.stats.getpopularphotos",
          {date=>$date,
           auth_token=>$flickrAuthToken});
 
say "Top Photo id is " . $gpp->{photos}->{photo}[0]->{id};
my $topphotoid = $gpp->{photos}->{photo}[0]->{id};
say "Top Photo view is " . $gpp->{photos}->{photo}[0]->{stats}->{views};
my $topphotoviews = $gpp->{photos}->{photo}[0]->{stats}->{views};
say "Top Photo Title is " . $gpp->{photos}->{photo}[0]->{title};
my $topphototitle = $gpp->{photos}->{photo}[0]->{title};
say "https:\/\/www.flickr.com\/photos\/mikebates\/$gpp->{photos}->{photo}[0]->{id}\/";
my $topphotourl = "https:\/\/www.flickr.com\/photos\/mikebates\/$gpp->{photos}->{photo}[0]->{id}\/";

# Get the token that we saved previously in order to authenticate:
my $session_filename = "/home/username/.stored_google_access.session";

# Be sure to edit the lines below to fill in your correct client
# id and client secret!
my $oauth2 = Net::Google::DataAPI::Auth::OAuth2->new(
 client_id => 'CLIENT_ID.apps.googleusercontent.com',
 client_secret => 'CLIENT SECRET',
    scope => ['http://spreadsheets.google.com/feeds/'],
    );

# Deserialize the file so we can thaw the session and reuse the refresh token
my $session = retrieve($session_filename);
 
my $restored_token = Net::OAuth2::AccessToken->session_thaw($session,
 auto_refresh => 1,
 profile => $oauth2->oauth2_webserver,
 );

$oauth2->access_token($restored_token);
# Now we can use this token to access the spreadsheets
# # in our account:
my $service = Net::Google::Spreadsheets->new( 
 auth => $oauth2);

my @spreadsheet = $service->spreadsheets();

# find a spreadsheet by title
  my $spreadsheet = $service->spreadsheet(
     { title => 'Flickr Activity - Start 2015'
     });

# find a worksheet by title
my $worksheet = $spreadsheet->worksheet(
   {title => 'RawStats'
    });

# add a row
   my $new_row = $worksheet->add_row(
       {
        date => $date,
        totalviews => $totalviews,
        photostreamviews => $photostreamviews,
        photoviews => $photoviews,
        setsviews => $setsviews,
        collectionsviews => $collectionsviews,
        galleriesviews => $galleriesviews,
        topphotoid => $topphotoid,
        topphotoviews => $topphotoviews,
        topphototitle => $topphototitle,
        topphotourl => $topphotourl,
      }
    );

Key lines added or changed,

Add lines 20, 21, and 22. They setup up the OAuth2 libraries and the Storable library.
Add lines 78 through 101, they setup access to the saved authorization file, and the connection to Google Drive and the spreadsheet.

Line 100 is the changed line 75 in the original program.  If you have everything in place to start adding Flickr data to your Google Spreadsheet.

Let me know if there is anything I missed or could improve upon here.

Saturday, February 21, 2015

Geocaching with the iPhone 5s

Well this weekend, February 14, 2015, I went out Geocaching with an iPhone 5s and the Geocaching app from geocaching.com and here is an assessment of my experience. It was pretty cool and it works with some effort.

My geocaching experience level would be considered novice and I wanted to do something along with my going out to take pictures (something to get me outside even more.)  So my first day out I found 5 of  9 that I searched for and second day 7 of  8, though 3 of these were ones I did not find the first day.  Like I said, novice level.  I had no clue how cache hiders really hid caches and learned the hard way by myself.

Using the Geocaching app was very easy, I select local "easy" to find caches and there were lots of them. The first cache I found ended up being in the knot of a tree and the second a magnetic key-holder on a lamppost.  Both of these were relatively easy.

The iPhone 5s as a GPS worked fairly well.  I found it would take me directly to the area of the cache and would actually try to take me straight to the cache.  I wondered if would take me directly to the point of the cache and it would not. Best I could do was one foot and even then it was off a bit.  I did validate the phone would not take me to a cache if there was no service which is what I expected. The approach that finally worked was to try and approach the cache from multiple directions and it would narrow down where I needed to go.  Experimenting worked out pretty well.

For the failures I had, well chock that up to inexperience.  I had no clue about guard rail caches and with some tips I got from a friend and a bit of research via Google and the geocaching.com site, day 2 was much better with a higher success rate.

One of the things I have learned is that the iPhone GPS can be a bit jittery.  It is impacted by metal buildings and overhead power lines.  I found using a regular GPS that these things were not a big a factor in reception.

Beautiful day. Cache I found in Longview, WA - GC4RPG8

Huge cache - Longview, WA - GC4RPG8


More things to learn though.   Next thing to figure out is how to download GPS information on caches to my GPS and use that instead of the preloaded caches.

You want to get outdoors, geocaching, photography and bird watching will get you there.  In the Pacific Northwest you have assume bad weather, rain, fog, mist,and wind.

Thoughts, please comment.