Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SP smarter params WIP #279

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

SP smarter params WIP #279

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
00da494
SP: better checks for setters and constructor params
breznak Feb 25, 2019
7405fd8
SP: add test for different constructor vs setter behavior
breznak Jan 11, 2018
fb2f89c
merge cleanup
breznak Feb 25, 2019
5849e1e
SP: warn about potential radius out of dimmensions
breznak Feb 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 46 additions & 27 deletions src/nupic/algorithms/SpatialPooler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,8 @@ SpatialPooler::SpatialPooler() {
}

SpatialPooler::SpatialPooler(
const vector<UInt> inputDimensions, const vector<UInt> columnDimensions,
const vector<UInt> inputDimensions,
const vector<UInt> columnDimensions,
UInt potentialRadius, Real potentialPct, bool globalInhibition,
Real localAreaDensity, Int numActiveColumnsPerInhArea,
UInt stimulusThreshold, Real synPermInactiveDec, Real synPermActiveInc,
Expand Down Expand Up @@ -134,15 +135,24 @@ UInt SpatialPooler::getNumInputs() const { return numInputs_; }
UInt SpatialPooler::getPotentialRadius() const { return potentialRadius_; }

void SpatialPooler::setPotentialRadius(UInt potentialRadius) {
NTA_CHECK(potentialRadius < numInputs_);
NTA_CHECK(potentialRadius < numInputs_/2);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of throwing error, this should truncate the radius to the number of inputs like the original constructor did: potentialRadius > numInputs_ ? numInputs_ : potentialRadius

Also, I think that the /2 is a mistake.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of throwing error, this should truncate the radius

I was thinking about this and I'd prefer not to (truncate). The user asks directly the API method (set*, constructor) to do something, and if the method cannot comply, I think we should let the user know ASAP, instead of silently modifying.

An example usecase is parameter optimization/swarming, where you'd waste cpu cycles evaluating range of values, while this might throw much earlier.
Of course, the negative effect is that many tests need to be updated to correct values.

Also, I think that the /2 is a mistake.

I think not. It's a radius, not diameter:

This parameter defines a square (or hyper square) area: a
column will have a max square potential pool with sides of
length (2 * potentialRadius + 1).

ok, on the other hand:

  • If the topology is one dimensional, and the potentialRadius is 5, this
    method will return an array containing 5 consecutive values centered on
    the index of the column (wrapping around if necessary).
  • If the topology is two dimensional (not implemented), and the
    potentialRadius is 5, the method should return an array containing 25
    '1's, where the exact indices are to be determined by the mapping from
    1-D index to 2-D position.

both from SpatialPooler.hpp, so either doc is wrong.

for(const auto dim: inputDimensions_) {
if(potentialRadius > (dim/2) +1) {
NTA_WARN << "potentialRadius >= one of the dimensions: " << dim;
}
}
potentialRadius_ = potentialRadius;
NTA_CHECK((UInt)(potentialPct_ * potentialRadius_) >= 1u) << "SP: at least 1 input synapse must be able to activate from potential pool.";
NTA_CHECK(stimulusThreshold_ <= potentialPct_ * potentialRadius) << "Stimulus threshold must be <= than the number of possibly active input synapses per segment.";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

potentialPct_ * potentialRadius is not the size of the potential pool, in the case where there are multiple dimensions. For this you can query the connections class?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is not the size of the potential pool, in the case where there are multiple dimensions

This is from the time when SP was essentially 1D and any nD inputs/SP were simulated by converting to 1D columnar array.

I was going to suggest changing Real potentialRadius to vector<Real> potentialRadii where size potentialRadii == num dimensions of inputs

For this you can query the connections class?

no comprende. afaik the Connections doesn't have a concept of potential pools(?) Do you suggest moving this functionality there?

}

Real SpatialPooler::getPotentialPct() const { return potentialPct_; }

void SpatialPooler::setPotentialPct(Real potentialPct) {
NTA_CHECK(potentialPct > 0.0f && potentialPct <= 1.0f);
potentialPct_ = potentialPct;
NTA_CHECK((UInt)(potentialPct_ * potentialRadius_) >= 1u) << "SP: at least 1 input synapse must be able to activate from potential pool.";
NTA_CHECK(stimulusThreshold_ <= potentialPct_ * potentialRadius_) << "Stimulus threshold must be <= than the number of possibly active input synapses per segment.";
}

bool SpatialPooler::getGlobalInhibition() const { return globalInhibition_; }
Expand All @@ -156,7 +166,7 @@ Int SpatialPooler::getNumActiveColumnsPerInhArea() const {
}

void SpatialPooler::setNumActiveColumnsPerInhArea(UInt numActiveColumnsPerInhArea) {
NTA_CHECK(numActiveColumnsPerInhArea > 0u && numActiveColumnsPerInhArea <= numColumns_); //TODO this boundary could be smarter
NTA_CHECK(numActiveColumnsPerInhArea > 0u && numActiveColumnsPerInhArea <= numColumns_);
numActiveColumnsPerInhArea_ = numActiveColumnsPerInhArea;
localAreaDensity_ = DISABLED; //MUTEX with localAreaDensity
}
Expand All @@ -172,6 +182,7 @@ void SpatialPooler::setLocalAreaDensity(Real localAreaDensity) {
UInt SpatialPooler::getStimulusThreshold() const { return stimulusThreshold_; }

void SpatialPooler::setStimulusThreshold(UInt stimulusThreshold) {
NTA_CHECK(stimulusThreshold_ <= potentialPct_ * potentialRadius_) << "Stimulus threshold must be <= than the number of possibly active input synapses per segment.";
stimulusThreshold_ = stimulusThreshold;
}

Expand Down Expand Up @@ -227,6 +238,8 @@ Real SpatialPooler::getSynPermActiveInc() const { return synPermActiveInc_; }
void SpatialPooler::setSynPermActiveInc(Real synPermActiveInc) {
NTA_CHECK( synPermActiveInc > connections::minPermanence );
NTA_CHECK( synPermActiveInc <= connections::maxPermanence );
if(synPermActiveInc >= synPermConnected_)
NTA_WARN << "SP: synPermActiveInc will learn in just one occurance.";
synPermActiveInc_ = synPermActiveInc;
}

Expand All @@ -237,6 +250,8 @@ Real SpatialPooler::getSynPermInactiveDec() const {
void SpatialPooler::setSynPermInactiveDec(Real synPermInactiveDec) {
NTA_CHECK( synPermInactiveDec >= connections::minPermanence );
NTA_CHECK( synPermInactiveDec <= connections::maxPermanence );
if(synPermInactiveDec >= connections::maxPermanence - synPermConnected_)
NTA_WARN << "SP: synPermInactiveDec will unlearn in just one example";
synPermInactiveDec_ = synPermInactiveDec;
}

Expand All @@ -247,6 +262,7 @@ Real SpatialPooler::getSynPermBelowStimulusInc() const {
void SpatialPooler::setSynPermBelowStimulusInc(Real synPermBelowStimulusInc) {
NTA_CHECK( synPermBelowStimulusInc > connections::minPermanence );
NTA_CHECK( synPermBelowStimulusInc <= connections::maxPermanence );
NTA_CHECK( synPermBelowStimulusInc <= synPermActiveInc_ * 0.5 ) << "Inactive growth should be << than for active synapses.";
synPermBelowStimulusInc_ = synPermBelowStimulusInc;
}

Expand Down Expand Up @@ -415,32 +431,32 @@ void SpatialPooler::initialize(
}
NTA_CHECK(numColumns_ > 0);
NTA_CHECK(numInputs_ > 0);
NTA_CHECK(inputDimensions_.size() == columnDimensions_.size());
NTA_CHECK(inputDimensions_.size() == columnDimensions_.size()) << "Input and SpatialPooler must have same dimensionality.";

NTA_CHECK((numActiveColumnsPerInhArea > 0 && localAreaDensity < 0) ||
(localAreaDensity > 0 && localAreaDensity <= MAX_LOCALAREADENSITY
&& numActiveColumnsPerInhArea < 0)
) << numActiveColumnsPerInhArea << " vs " << localAreaDensity;
numActiveColumnsPerInhArea_ = numActiveColumnsPerInhArea;
localAreaDensity_ = localAreaDensity;
NTA_CHECK((numActiveColumnsPerInhArea > 0 || localAreaDensity > 0.0f) && !(localAreaDensity > 0.0f && numActiveColumnsPerInhArea > 0)) << "exactly one of these must be enabled";
if(numActiveColumnsPerInhArea > 0) {
setNumActiveColumnsPerInhArea(numActiveColumnsPerInhArea);
} else {
setLocalAreaDensity(localAreaDensity);
}

rng_ = Random(seed);

potentialRadius_ = potentialRadius > numInputs_ ? numInputs_ : potentialRadius;
NTA_CHECK(potentialPct > 0 && potentialPct <= 1);
potentialPct_ = potentialPct;
setPotentialRadius(potentialRadius);
setPotentialPct(potentialPct);
globalInhibition_ = globalInhibition;
stimulusThreshold_ = stimulusThreshold;
synPermInactiveDec_ = synPermInactiveDec;
synPermActiveInc_ = synPermActiveInc;
synPermBelowStimulusInc_ = synPermConnected / 10.0f;
setStimulusThreshold(stimulusThreshold);
NTA_CHECK(0.0 < synPermConnected && synPermConnected < 1.0);
synPermConnected_ = synPermConnected;
minPctOverlapDutyCycles_ = minPctOverlapDutyCycles;
dutyCyclePeriod_ = dutyCyclePeriod;
boostStrength_ = boostStrength;
spVerbosity_ = spVerbosity;
wrapAround_ = wrapAround;
updatePeriod_ = 50u;
setSynPermInactiveDec(synPermInactiveDec);
setSynPermActiveInc(synPermActiveInc);
setSynPermBelowStimulusInc(synPermConnected / 10.0f);
setMinPctOverlapDutyCycles(minPctOverlapDutyCycles);
setDutyCyclePeriod(dutyCyclePeriod);
setBoostStrength(boostStrength);
setSpVerbosity(spVerbosity);
setWrapAround(wrapAround);
setUpdatePeriod(50u);
initConnectedPct_ = 0.5f;
iterationNum_ = 0u;
iterationLearnNum_ = 0u;
Expand Down Expand Up @@ -584,8 +600,7 @@ vector<UInt> SpatialPooler::initMapPotential_(UInt column, bool wrapAround) {
columnInputs.push_back(input);
}
} else {
for (UInt input :
Neighborhood(centerInput, potentialRadius_, inputDimensions_)) {
for (UInt input : Neighborhood(centerInput, potentialRadius_, inputDimensions_)) {
columnInputs.push_back(input);
}
}
Expand All @@ -601,13 +616,17 @@ Real SpatialPooler::initPermConnected_() {
Real p =
synPermConnected_ + (Real)((connections::maxPermanence - synPermConnected_) * rng_.getReal64());

return round5_(p);
p = round5_(p);
NTA_ASSERT(p >= synPermConnected_);
return p;
}


Real SpatialPooler::initPermNonConnected_() {
Real p = (Real)(synPermConnected_ * rng_.getReal64());
return round5_(p);
p = round5_(p);
NTA_ASSERT(p < synPermConnected_);
return p;
}


Expand Down
20 changes: 13 additions & 7 deletions src/nupic/algorithms/SpatialPooler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ namespace algorithms {
namespace spatial_pooler {

using namespace std;
static const int DISABLED = -1; //value denoting a feature is disabled
static const int DISABLED = 0; //value denoting a feature is disabled

/**
* CLA spatial pooler implementation in C++.
Expand Down Expand Up @@ -70,14 +70,20 @@ class SpatialPooler : public Serializable
{
public:
SpatialPooler();
SpatialPooler(const vector<UInt> inputDimensions, const vector<UInt> columnDimensions,
UInt potentialRadius = 16u, Real potentialPct = 0.5f,
bool globalInhibition = true, Real localAreaDensity = DISABLED,
SpatialPooler(const vector<UInt> inputDimensions,
const vector<UInt> columnDimensions,
UInt potentialRadius = 16u,
Real potentialPct = 0.5f,
bool globalInhibition = true,
Real localAreaDensity = DISABLED,
Int numActiveColumnsPerInhArea = 10u,
UInt stimulusThreshold = 0u, Real synPermInactiveDec = 0.008f,
Real synPermActiveInc = 0.05f, Real synPermConnected = 0.1f,
UInt stimulusThreshold = 0u,
Real synPermInactiveDec = 0.008f,
Real synPermActiveInc = 0.05f,
Real synPermConnected = 0.1f,
Real minPctOverlapDutyCycles = 0.001f,
UInt dutyCyclePeriod = 1000u, Real boostStrength = 0.0f,
UInt dutyCyclePeriod = 1000u,
Real boostStrength = 0.0f,
Int seed = 1, UInt spVerbosity = 0u, bool wrapAround = true);

virtual ~SpatialPooler() {}
Expand Down
29 changes: 27 additions & 2 deletions src/test/unit/algorithms/SpatialPoolerTest.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2005,14 +2005,39 @@ TEST(SpatialPoolerTest, testSerialization2) {
}
}
}

cout << "Timing for SpatialPooler serialization (smaller is better):" << endl;
cout << "Stream: " << testTimer.getElapsed() << endl;

remove("outC.stream");
}


TEST(SpatialPoolerTest, testDifferentConstructorVsSetterBehavior)
{
/** this test exposes wrong behavior, where SP created via constructor
behaves differently to a SP via setters (setXXX()), both with the same
params.
*/
SpatialPooler spConstruct{std::vector<UInt>{10} /* input*/, std::vector<UInt>{2048}/* SP output cols XXX sensitive*/,
/*pot radius*/ 20, //each col sees
/*pot pct*/ 0.5, //XXX sensitive
/*global inhibition*/ false, //XXX sensitive
/*Real localAreaDensity=*/0.02, //2% active cols
/*UInt numActiveColumnsPerInhArea=*/0, //mutex with above ^^ //XXX sensitive
};

SpatialPooler sp{std::vector<UInt>{10} /* input*/, std::vector<UInt>{2048}/* SP output cols */};
sp.setPotentialRadius(20);
sp.setPotentialPct(0.5);
sp.setGlobalInhibition(false);
sp.setLocalAreaDensity(0.02); //2% active cols
sp.setNumActiveColumnsPerInhArea(0); //mutex with above ^^


// EXPECT_EQ(spConstruct, sp); //FIXME how compare 2 SP
check_spatial_eq(spConstruct, sp);
}


TEST(SpatialPoolerTest, testConstructorVsInitialize) {
// Initialize SP using the constructor
SpatialPooler sp1(
Expand Down