I made a 2D flocking algorithm for my Advanced Game AI class, but the code can be easily converted to work in 3D as well. Using a mix of the 3 main behaviors in flocking (cohesion, alignment and avoidance), a few extra behaviors including group flocking and obstacle avoidance, plus composite to put it all together, we get a decently expandable flocking algorithm. Above is an example of it in action, and at the bottom you can see an example of the flocks avoiding each other as well. You can access the code here.
Flock
The Flock.cs script has two main functions, Update() and GetNearbyObjects().
Update: Get all the data from the behaviors, use it to calculate the movement and tell the agent the final move value.
GetNearbyObjects: Uses physics overlap circle to check for all other objects nearby.
Behaviors
Flocking is made up of 3 behaviors: cohesion, alignment and avoidance. There's also composite which takes data from all 3 of these and calculates the final movement values. The 3 main behaviors are pretty similar code wise wise, mainly being that they all check to make sure there are neighbors, otherwise it'll just return nothing as well as checking to see if any special filtering was done (This is used for things like finding their own flock as well as obstacle avoidance).
Cohesion: Cohesion adds up all the positions and gets the average plus the offset.
Alignment: Alignment adds up all the directions the agents are facing and gets the average.
Avoidance: Avoidance calculates all the differences in positions between the agent and other objects surrounding it, which is then averaged by the number of things the agents avoiding.
Composite: Composite takes all this data and calculates the movement off of it as well as the weight set to each behavior. There's also a check just to make sure the behavior doesn't exceed that amount of weight it should have.
Check for Specific Flock
To keep track of agents that are part of their flock, they just check all nearby objects and store all the ones that have the same flock type in a new list. Each agent gets designated a flock at the start of runtime through the Flock.cs script. The Flock.cs script is attached the an empty gameObject that holds all the flocks. The flock type is decided by whatever prefab you use in the Flock.cs script in the inspector.
Obstacle Avoidance
To avoid obstacles, there's a Obstacle Avoidance Filter Scriptable Object. The scriptable object has it so you can set what layer you want the flock to avoid, and the script will get a list of all objects on that layer based on objects nearby. From there this object is added to the Avoidance Behavior Scriptable Object where all the math happens.
You can see from the gif at the start of this devblog, the obstacle avoidance working for walls. Since it's based on layers, as long as you give each flock it's own layer you can make flocks avoid each other as well.
Inspector
Lastly, through the inspector, the user can adjust a wide array of values for their flocks.
Behavior: A composite behavior scriptable object that has all the behaviors you want the flock to take into account. The composite behavior has a list of all the behaviors and their weight values.
Starting Count: Number of agents you want in the flock.
Agent Density: Used to calculate spawning.
Drive Factor: A value that gets multiplied to the movement to speed up the agent.
Max Speed: Max speed the agent can go to after all the calculations have been done.
Neighbor Radius: How far the agent is checking for things.
Avoidance Radius Multiplier: Used to calculate avoidance.
