#Risk Scoring System Plan

Streamlined scoring system with 7 core risk factors. High inbound volume is rewarded, high outbound volume is penalized.

#Overview

The risk scoring system analyzes phone number usage patterns to identify potential spam or abusive behavior that could lead to carrier filtering or account suspension. The system calculates a risk score from 0-100, where higher scores indicate higher risk.

#Risk Levels

• Low Risk (0-29): Healthy usage patterns
• Medium Risk (30-59): Some concerning patterns that should be monitored
• High Risk (60-100): Significant risk indicators present

#Risk Factors

#1. Conversations Per Day (Volume + Direction)

Impact: HIGH

• Metrics:
  • Average unique conversations initiated per day (30-day window)
  • Percentage of inbound-initiated vs outbound-initiated conversations
  • Total conversation volume
• Scoring Logic:

  Base Volume Scoring (for OUTBOUND-initiated conversations only):

  • > 50 outbound conversations/day: +30 points (VERY HIGH risk)
  • 40-50 outbound conversations/day: +20 points (HIGH risk)
  • 30-40 outbound conversations/day: +8 points (LOW risk)
  • < 30 outbound conversations/day: 0 points (OK)

  Inbound Volume Bonus (high inbound volume is GOOD):

  • ≥ 100 inbound conversations/day: 20 points (EXCELLENT - thriving business)
  • 50-100 inbound conversations/day: 15 points (GREAT - high organic demand)
  • 30-50 inbound conversations/day: 10 points (GOOD - solid organic traffic)
  • 15-30 inbound conversations/day: 5 points (DECENT - healthy inbound)
  • < 15 inbound conversations/day: 0 points (NEUTRAL)

  Inbound Percentage Modifier (gradient - higher inbound % = better):

Why it matters: High conversation volume is only risky when it's outbound (cold outreach). High INBOUND conversation volume is fantastic - it means customers are reaching out to you! The scoring separates these: outbound volume adds risk, inbound volume gives bonuses. A business with 100+ inbound conversations/day is thriving, not spamming.

Note: Always evaluated.

#2. Total Contacts

Impact: MEDIUM

• Metric: Total unique contacts ever messaged from this number
• Target: ≤1,500 contacts
• Scoring:
  • > 2,000 contacts: +15 points (HIGH risk)
  • 1,500-2,000 contacts: +8 points (WARNING)
  • < 1,500 contacts: 0 points (OK)

Why it matters: Accumulating thousands of contacts indicates potential broadcast/spam usage.

Note: Always evaluated.

#3. Reply Rate

Impact: VERY HIGH

• Metric: Percentage of conversations that received meaningful replies (3+ inbound messages)
• Definition: A conversation is only considered to have a "reply" when 3 or more inbound messages have been received. Fewer than 3 inbound messages does not count as a reply.
• Target: ≥40%
• Scoring:
  • < 20% reply rate: +35 points (VERY HIGH risk)
  • 20-30% reply rate: +20 points (HIGH risk)
  • 30-40% reply rate: +8 points (WARNING)
  • 40-50% reply rate: 0 points (OK)
  • ≥ 50% reply rate: 5 points (BONUS - excellent engagement)

Why it matters: Low reply rates indicate messages aren't wanted or are perceived as spam. This is one of the strongest spam signals. The 3+ message threshold ensures we're measuring genuine engagement, not just a single acknowledgment.

Evaluation Threshold: Only evaluated once there are ≥10 total conversations (need sufficient sample size for meaningful rate).

#4. Conversation Engagement Quality

Impact: HIGH

This metric evaluates the quality of engagement within conversations, focusing on inbound activity.

• Metrics:
  • Average inbound messages per conversation
  • Inbound-to-outbound message ratio within conversations
• Scoring:

  Inbound Message Count (per conversation):

  • < 3 inbound messages/conversation: +25 points (HIGH risk - low engagement)
  • ≥ 3 inbound messages/conversation: 0 points (OK - engaged recipients)

  Inbound/Outbound Ratio (within conversations):

  • < 20% inbound ratio: +20 points (HIGH risk - one-way communication)
  • 20-30% inbound ratio: +10 points (WARNING - weak engagement)
  • ≥ 30% inbound ratio: 0 points (OK - healthy back-and-forth)

Why it matters: Shallow conversations with minimal inbound responses indicate spam. Real conversations have genuine back-and-forth exchanges where recipients actively participate.

Evaluation Thresholds:

• Inbound Message Count: Evaluated once there are ≥3 conversations (need enough data to calculate averages)
• Inbound/Outbound Ratio: Evaluated once there are ≥10 total messages across all conversations

#5. Burst Detection - New Conversations

Impact: VERY HIGH

• Metric: Maximum new conversations started in any single hour (30-day window)
• Target: ≤15-20 new conversations/hour (~1 per 3-4 minutes, comfortably human)
• Scoring:
  • > 60 new conversations/hour: +40 points (CRITICAL - obvious automation)
  • 40-60 new conversations/hour: +30 points (VERY HIGH risk - automation territory)
  • 25-40 new conversations/hour: +20 points (HIGH risk - suspicious pace)
  • 15-25 new conversations/hour: +10 points (WARNING - power user)
  • < 15 new conversations/hour: 0 points (OK - human pace, ~1 per 4 minutes)

Why it matters: Starting many new conversations rapidly is a hallmark of spam campaigns and mass outreach automation. This is more risky than message bursts within existing conversations. A human pace is roughly 1 new conversation every 3-4 minutes.

Note: Always evaluated. Looks at the maximum spike in any single hour over the 30-day window.

#6. Burst Detection - Messages Within Conversations

Impact: HIGH

• Metric: Maximum messages sent in any single hour (30-day window)
• Target: <100 messages/hour (comfortably human)
• Scoring:
  • > 200 messages/hour: +30 points (VERY HIGH risk - obvious automation)
  • 150-200 messages/hour: +20 points (HIGH risk - automation territory)
  • 100-150 messages/hour: +10 points (WARNING - power user)
  • < 100 messages/hour: 0 points (OK - human pace)

Why it matters: Sustained high-velocity sending is impossible for humans and indicates automation/botting. Over 200 messages per hour is a major red flag. However, this is less concerning than new conversation bursts if messages are going to existing, engaged contacts.

Note: Always evaluated. Looks at the maximum spike in any single hour over the 30-day window.

#7. Message Similarity (SimHash Variance)

Impact: MEDIUM (Only computed with compute=true)

• Metric: Percentage of messages with similar simhash values (indicating templated/repetitive content)
• Minimum Message Length: Only messages with ≥50 characters are considered. Short messages like "thanks", "ok", "got it" are excluded since they're natural and common.
• Target: <50% similar messages (among qualifying messages)
• Scoring:
  • > 70% similar messages: +15 points (HIGH risk - robotic/templated)
  • 50-70% similar messages: +8 points (WARNING)
  • < 50% similar messages: 0 points (OK)
  • < 30% similar messages: 2 points (BONUS - natural variation)

Why it matters: Repetitive templated messages are a spam hallmark. Natural conversations vary in content and length. SimHash provides a lightweight way to detect this without expensive text analysis.

Evaluation Threshold: Only evaluated if there are ≥20 qualifying messages (messages with ≥50 characters).

#What is Good vs. Bad

#✅ Good (Low Risk) Patterns

• High inbound conversation volume - 50+ inbound conversations/day is fantastic (thriving business!)
• Reply rates above 50% - recipients are genuinely engaged (3+ inbound messages = reply)
• High inbound message count (≥3 per conversation) - real engagement
• Healthy inbound ratio (≥30%) - genuine back-and-forth dialogue
• Low outbound conversation volume - <30 outbound conversations/day
• High inbound-initiated percentage - 70%+ inbound (gradient: higher = better)
• Natural message variance - <50% similar messages among longer messages (≥50 chars)
• Human sending pace - <100 messages/hour, <15 new conversations/hour
• Reasonable contact list - <1,500 contacts

#❌ Bad (High Risk) Patterns

• Low reply rates <20% - recipients aren't engaging (spam indicator)
• Low inbound engagement - <3 inbound messages per conversation
• One-way communication - <20% inbound ratio in conversations
• High OUTBOUND conversation volume - >50 outbound conversations/day
• Heavy cold outreach - <20% inbound-initiated conversations
• New conversation bursts - >25 new conversations/hour
• Message bursts - >200 messages/hour sustained
• Robotic consistency - >70% similar messages among longer messages (≥50 chars)
• Large contact lists - >2,000 contacts

#Scoring Penalties Summary

Maximum possible penalty: ~230 points (capped at 100)

Maximum possible bonus: -42 points (floor at 0)

#Data Collection

#Standard Metrics (Always Computed)

These are computed every time, using efficient batch queries:

• Outbound conversations per day (volume)
• Inbound conversations per day (volume - used for bonus)
• Inbound vs outbound conversation initiation percentage (gradient)
• Total contacts
• Reply rate (conversations with 3+ inbound messages)
• Inbound message count per conversation
• Inbound/outbound message ratio within conversations
• Max new conversations per hour (burst detection)
• Max messages per hour (burst detection)

#Expensive Metrics (Computed with compute=true)

These require expensive database joins and are cached for 24 hours:

• Message similarity (simhash clustering analysis, only messages ≥50 characters)

For small organizations (≤5 numbers): Expensive metrics are auto-computed if not cached.

For large organizations (>5 numbers): Must explicitly request compute=true, with rate limiting (5-hour throttle between computes).

#API Endpoint

GET /organizations/:organizationId/analytics/risk-tolerance

Query Parameters:

• compute=true - Force compute expensive metrics (rate-limited to once per 5 hours)

Response:

{
  "phone_numbers": [
    {
      "phone_number": "+15555551234",
      "allocation_id": "alloc_abc123",
      "risk_score": 12,
      "risk_level": "low",
      "metrics": {
        "avg_outbound_conversations_per_day": 15.2,
        "avg_inbound_conversations_per_day": 45.8,
        "inbound_initiated_percentage": 75.1,
        "total_contacts": 890,
        "reply_rate": 52.5,
        "avg_inbound_messages_per_conversation": 4.2,
        "inbound_outbound_message_ratio": 0.38,
        "max_new_conversations_per_hour": 12,
        "max_messages_per_hour": 85,
        "message_similarity_percentage": 32.1
      },
      "risk_factors": {
        "outboundConversationsPerDay": {
          "value": 15.2,
          "score": 0,
          "status": "ok",
          "impact": "high"
        },
        "inboundConversationsPerDay": {
          "value": 45.8,
          "score": -10,
          "status": "bonus",
          "impact": "high"
        },
        "inboundPercentageModifier": {
          "value": 0.751,
          "score": -8,
          "status": "bonus",
          "impact": "high"
        },
        "replyRate": {
          "value": 0.525,
          "score": -5,
          "status": "bonus",
          "impact": "very_high"
        },
        "newConversationBursts": {
          "value": 12,
          "score": 0,
          "status": "ok",
          "impact": "very_high"
        }
      },
      "daily_data": {
        "conversations": [],
        "messages": []
      }
    }
  ],
  "period_days": 30,
  "calculated_at": 1701388800000,
  "computed_with_expensive_metrics": true
}

#Best Practices

#To Maintain Low Risk Scores:

1. Focus on engagement - Only message people likely to respond (aim for >50% reply rate with 3+ inbound messages)
2. Encourage conversations - Send messages that prompt replies, aim for ≥3 inbound messages per conversation
3. Maintain healthy ratios - Keep inbound/outbound message ratio above 30%
4. Limit conversation volume - Stay under 30 conversations/day
5. Avoid new conversation bursts - Don't start more than 15 new conversations/hour (~1 per 4 minutes)
6. Avoid message bursts - Keep under 100 messages/hour
7. Personalize messages - Vary content (keep simhash similarity under 50%)
8. Grow organically - Higher inbound-initiated conversations = lower risk (gradient: 50%+ is good, 90%+ is excellent)
9. Quality over quantity - Better to have 10 engaged conversations than 100 ignored ones
10. Keep contact lists reasonable - Aim for under 1,500 contacts per number

#Red Flags to Avoid:

• Consistent <20% reply rates (conversations with 3+ inbound messages)
• Less than 3 inbound messages per conversation on average
• <20% inbound/outbound message ratio (one-way communication)
• 50 conversations per day
• Starting 25+ new conversations in a single hour
• Sending 200+ messages in a single hour
• Using similar templates for 70%+ of messages
• Having <20% inbound-initiated conversations
• Building contact lists of 2,000+ people per number

#Technical Implementation Notes

#Allocation Type Filtering

Only dedicated is included in risk scoring. Trial/shared/2fa numbers are excluded as they don't have real usage patterns suitable for risk analysis.

#Caching Strategy

• Cache key: risk-tolerance:v4.1:{organizationId}
• TTL: 24 hours
• Cache invalidation: Automatic expiration
• Throttling: 5-hour minimum between forced recomputes

#Sequential Processing

For organizations with >5 numbers, queries are processed sequentially (one allocation at a time) to avoid Cloudflare Workers CPU limits. Small orgs use batch queries for speed.

#Related Metrics

#Conversations Table

The system uses a conversations table that pre-aggregates:

• total_conversations: Count of unique external_id per allocation
• conversations_with_replies: Count where inbound_count >= 3 (3+ inbound messages = meaningful reply)
• initiated_by: 'inbound' or 'outbound' (who sent first message)

#Messages Table Fields Used

• allocation_id: Phone number allocation
• external_id: Unique conversation identifier
• direction: 'inbound' or 'outbound'
• time_sent: Message timestamp
• text_length: Character count
• simhash_text: Template similarity hash
• contact_id: Unique contact identifier
• protocol: 'sms', 'imessage', or 'rcs'